[jira] [Resolved] (MAPREDUCE-6373) The logger reports total input paths but it is referring to input files
[
https://issues.apache.org/jira/browse/MAPREDUCE-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K resolved MAPREDUCE-6373.
--
Resolution: Fixed
Fix Version/s: 2.8.0
Thanks [~bibinchundatt].
Committed to trunk and branch-2.
> The logger reports total input paths but it is referring to input files
> ---
>
> Key: MAPREDUCE-6373
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6373
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Andi Chirita Amdocs
>Assignee: Bibin A Chundatt
>Priority: Trivial
> Labels: newbie
> Fix For: 2.8.0
>
> Attachments: 0002-MAPREDUCE-6373.patch, MAPREDUCE-6373.patch
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> The log message in the FileInputFormat is misleading :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 6
> {code}
> There is only 1 input path and 6 input files so the log message should be :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input files to
> process : 6
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6373) The logger reports total input paths but it is referring to input files
[
https://issues.apache.org/jira/browse/MAPREDUCE-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-6373:
-
Hadoop Flags: Reviewed
+1, looks good to me.
> The logger reports total input paths but it is referring to input files
> ---
>
> Key: MAPREDUCE-6373
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6373
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Andi Chirita Amdocs
>Assignee: Bibin A Chundatt
>Priority: Trivial
> Labels: newbie
> Attachments: 0002-MAPREDUCE-6373.patch, MAPREDUCE-6373.patch
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> The log message in the FileInputFormat is misleading :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 6
> {code}
> There is only 1 input path and 6 input files so the log message should be :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input files to
> process : 6
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6316) Task Attempt List entries should link to the task overview
[
https://issues.apache.org/jira/browse/MAPREDUCE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591280#comment-14591280
]
Hadoop QA commented on MAPREDUCE-6316:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 18m 14s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 45s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 52s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 26s | The applied patch generated 1
new checkstyle issues (total was 43, now 43). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 33s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | mapreduce tests | 9m 3s | Tests failed in
hadoop-mapreduce-client-app. |
| {color:green}+1{color} | mapreduce tests | 5m 55s | Tests passed in
hadoop-mapreduce-client-hs. |
| {color:green}+1{color} | yarn tests | 1m 57s | Tests passed in
hadoop-yarn-common. |
| | | 60m 44s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.mapreduce.v2.app.webapp.TestBlocks |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740223/MAPREDUCE-6316.v7.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 74351af |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5811/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5811/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-hs.txt
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5811/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| hadoop-mapreduce-client-hs test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5811/artifact/patchprocess/testrun_hadoop-mapreduce-client-hs.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5811/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5811/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5811/console |
This message was automatically generated.
> Task Attempt List entries should link to the task overview
> --
>
> Key: MAPREDUCE-6316
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6316
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: Siqi Li
>Assignee: Siqi Li
> Labels: BB2015-05-TBR
> Attachments: AM attempt page.png, AM task page.png, All Attempts
> page.png, MAPREDUCE-6316.v1.patch, MAPREDUCE-6316.v2.patch,
> MAPREDUCE-6316.v3.patch, MAPREDUCE-6316.v4.patch, MAPREDUCE-6316.v5.patch,
> MAPREDUCE-6316.v6.patch, MAPREDUCE-6316.v7.patch, Task Overview page.png
>
>
> Typical workflow is to click on the list of failed attempts. Then you want to
> look at the counters, or the list of attempts of just one task in general. If
> each entry task attempt id linked the task id portion of it back to the task,
> we would not have to go through the list of tasks to search for the task.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591172#comment-14591172
]
Hadoop QA commented on MAPREDUCE-6400:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 5s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 41s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 56s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 0s | The applied patch generated 1
new checkstyle issues (total was 0, now 1). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 31s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 15s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | mapreduce tests | 1m 33s | Tests failed in
hadoop-mapreduce-client-core. |
| {color:red}-1{color} | mapreduce tests | 105m 34s | Tests failed in
hadoop-mapreduce-client-jobclient. |
| | | 147m 53s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.mapred.TestLineRecordReader |
| | hadoop.mapreduce.lib.input.TestLineRecordReader |
| | hadoop.mapred.TestMiniMRChildTask |
| | hadoop.mapred.TestMerge |
| | hadoop.mapred.TestReduceFetch |
| | hadoop.mapreduce.v2.TestUberAM |
| | hadoop.mapred.TestReduceFetchFromPartialMem |
| | hadoop.mapred.TestMRIntermediateDataEncryption |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740193/MAPREDUCE-6400-002.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / cc43288 |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5810/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5810/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-jobclient test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5810/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5810/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5810/console |
This message was automatically generated.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5002) AM could potentially allocate a reduce container to a map attempt
[ https://issues.apache.org/jira/browse/MAPREDUCE-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-5002: Status: Patch Available (was: Open) > AM could potentially allocate a reduce container to a map attempt > - > > Key: MAPREDUCE-5002 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5002 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 0.23.7, 2.0.3-alpha >Reporter: Jason Lowe >Assignee: Chang Li > Attachments: MAPREDUCE-5002.1.patch > > > As discussed in MAPREDUCE-4982, after MAPREDUCE-4893 it is theoretically > possible for the AM to accidentally assign a reducer container to a map > attempt if the AM doesn't find a reduce attempt actively looking for the > container (e.g.: the RM accidentally allocated too many reducer containers). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5002) AM could potentially allocate a reduce container to a map attempt
[ https://issues.apache.org/jira/browse/MAPREDUCE-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-5002: Attachment: MAPREDUCE-5002.1.patch > AM could potentially allocate a reduce container to a map attempt > - > > Key: MAPREDUCE-5002 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5002 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.0.3-alpha, 0.23.7 >Reporter: Jason Lowe >Assignee: Chang Li > Attachments: MAPREDUCE-5002.1.patch > > > As discussed in MAPREDUCE-4982, after MAPREDUCE-4893 it is theoretically > possible for the AM to accidentally assign a reducer container to a map > attempt if the AM doesn't find a reduce attempt actively looking for the > container (e.g.: the RM accidentally allocated too many reducer containers). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6402) Application Master listen on all IPs
[ https://issues.apache.org/jira/browse/MAPREDUCE-6402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591096#comment-14591096 ] Peter Shi commented on MAPREDUCE-6402: -- I think there are 2 approaches: 1) bind the same ip with NodeManager, it is easy to implement, but the constraint is NM and AM should be in the same subnet. 2) configuration subnet for AM, like "192.168.0.0/8", and the AM try to bind the ip belongs to the subnet in the container. > Application Master listen on all IPs > > > Key: MAPREDUCE-6402 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6402 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: applicationmaster >Affects Versions: 2.7.0 >Reporter: Peter Shi >Assignee: Varun Saxena > > The application master listening on all IPs, which may cause security > problems. Similar issue: MAPREDUCE-5938 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6405) NullPointerException in App Attempts page
[
https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591069#comment-14591069
]
Hadoop QA commented on MAPREDUCE-6405:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 17s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 49s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 57s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 38s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 39s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 8s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 9m 10s | Tests passed in
hadoop-mapreduce-client-app. |
| | | 47m 36s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740179/MAPREDUCE-6405.v1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 1a169a2 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5809/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5809/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5809/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5809/console |
This message was automatically generated.
> NullPointerException in App Attempts page
> -
>
> Key: MAPREDUCE-6405
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Siqi Li
>Assignee: Siqi Li
> Attachments: MAPREDUCE-6405.v1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-3384) Add warning message for underflow or overflow
[
https://issues.apache.org/jira/browse/MAPREDUCE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590808#comment-14590808
]
Hadoop QA commented on MAPREDUCE-3384:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 13s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 35s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 39s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 44s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 2m 59s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 1m 37s | Tests passed in
hadoop-mapreduce-client-core. |
| {color:red}-1{color} | mapreduce tests | 124m 12s | Tests failed in
hadoop-mapreduce-client-jobclient. |
| {color:green}+1{color} | mapreduce tests | 2m 17s | Tests passed in
hadoop-mapreduce-client-nativetask. |
| | | 169m 50s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-mapreduce-client-core |
| Failed unit tests | hadoop.mapred.TestMRIntermediateDataEncryption |
| | hadoop.mapred.TestMerge |
| | hadoop.fs.TestFileSystem |
| | hadoop.mapred.TestMiniMRClientCluster |
| | hadoop.mapred.TestReduceFetch |
| | hadoop.mapred.TestReduceFetchFromPartialMem |
| Timed out tests |
org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740119/MAPREDUCE-3384-004.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 6e3fcff |
| whitespace |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5808/artifact/patchprocess/whitespace.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5808/artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5808/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-jobclient test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5808/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
|
| hadoop-mapreduce-client-nativetask test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5808/artifact/patchprocess/testrun_hadoop-mapreduce-client-nativetask.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5808/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5808/console |
This message was automatically generated.
> Add warning message for underflow or overflow
> -
>
> Key: MAPREDUCE-3384
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3384
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: JiangKai
>Assignee: Brahma Reddy Battula
> Attachments: MAPREDUCE-3384-002.patch, MAPREDUCE-3384-003.patch,
> MAPREDUCE-3384-004.patch, MAPREDUCE-3384.patch
>
>
> When we call the function reduce() of LongSumReducer,the result may overflow
> or underflow.
> We should have a warning message to users if overflow/underflow occurs for
> all these classes,
> {code:xml}
> /java/org/apache/hadoop/mapred/lib/LongSumReducer.java
> .../apache/hadoop/mapreduce/lib/reduce/IntSumReducer.java
> .../apache/hadoop/mapreduce/lib/reduce/LongSumReducer.java
> .../test/java/org/apache/hadoop/fs/AccumulatingReducer.java
> .../hadoop/mapred/nativetask/kvtest/HashSumReducer.java
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-5002) AM could potentially allocate a reduce container to a map attempt
[ https://issues.apache.org/jira/browse/MAPREDUCE-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li reassigned MAPREDUCE-5002: --- Assignee: Chang Li > AM could potentially allocate a reduce container to a map attempt > - > > Key: MAPREDUCE-5002 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5002 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.0.3-alpha, 0.23.7 >Reporter: Jason Lowe >Assignee: Chang Li > > As discussed in MAPREDUCE-4982, after MAPREDUCE-4893 it is theoretically > possible for the AM to accidentally assign a reducer container to a map > attempt if the AM doesn't find a reduce attempt actively looking for the > container (e.g.: the RM accidentally allocated too many reducer containers). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6316) Task Attempt List entries should link to the task overview
[ https://issues.apache.org/jira/browse/MAPREDUCE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590701#comment-14590701 ] Siqi Li commented on MAPREDUCE-6316: Hi [~jira.shegalov], I have uploaded patch v7, which added a unit test for attempts page, and I have also tested it with additional patch from MAPREDUCE-6405 on my local machine, and it works properly. > Task Attempt List entries should link to the task overview > -- > > Key: MAPREDUCE-6316 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6316 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Siqi Li >Assignee: Siqi Li > Labels: BB2015-05-TBR > Attachments: AM attempt page.png, AM task page.png, All Attempts > page.png, MAPREDUCE-6316.v1.patch, MAPREDUCE-6316.v2.patch, > MAPREDUCE-6316.v3.patch, MAPREDUCE-6316.v4.patch, MAPREDUCE-6316.v5.patch, > MAPREDUCE-6316.v6.patch, MAPREDUCE-6316.v7.patch, Task Overview page.png > > > Typical workflow is to click on the list of failed attempts. Then you want to > look at the counters, or the list of attempts of just one task in general. If > each entry task attempt id linked the task id portion of it back to the task, > we would not have to go through the list of tasks to search for the task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6316) Task Attempt List entries should link to the task overview
[ https://issues.apache.org/jira/browse/MAPREDUCE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li updated MAPREDUCE-6316: --- Status: Patch Available (was: Open) > Task Attempt List entries should link to the task overview > -- > > Key: MAPREDUCE-6316 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6316 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Siqi Li >Assignee: Siqi Li > Labels: BB2015-05-TBR > Attachments: AM attempt page.png, AM task page.png, All Attempts > page.png, MAPREDUCE-6316.v1.patch, MAPREDUCE-6316.v2.patch, > MAPREDUCE-6316.v3.patch, MAPREDUCE-6316.v4.patch, MAPREDUCE-6316.v5.patch, > MAPREDUCE-6316.v6.patch, MAPREDUCE-6316.v7.patch, Task Overview page.png > > > Typical workflow is to click on the list of failed attempts. Then you want to > look at the counters, or the list of attempts of just one task in general. If > each entry task attempt id linked the task id portion of it back to the task, > we would not have to go through the list of tasks to search for the task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6316) Task Attempt List entries should link to the task overview
[ https://issues.apache.org/jira/browse/MAPREDUCE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li updated MAPREDUCE-6316: --- Attachment: MAPREDUCE-6316.v7.patch > Task Attempt List entries should link to the task overview > -- > > Key: MAPREDUCE-6316 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6316 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Siqi Li >Assignee: Siqi Li > Labels: BB2015-05-TBR > Attachments: AM attempt page.png, AM task page.png, All Attempts > page.png, MAPREDUCE-6316.v1.patch, MAPREDUCE-6316.v2.patch, > MAPREDUCE-6316.v3.patch, MAPREDUCE-6316.v4.patch, MAPREDUCE-6316.v5.patch, > MAPREDUCE-6316.v6.patch, MAPREDUCE-6316.v7.patch, Task Overview page.png > > > Typical workflow is to click on the list of failed attempts. Then you want to > look at the counters, or the list of attempts of just one task in general. If > each entry task attempt id linked the task id portion of it back to the task, > we would not have to go through the list of tasks to search for the task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590583#comment-14590583 ] Ray Chiang commented on MAPREDUCE-6376: --- [~jlowe], this JHS speedup should have the least side effects (unlike MAPREDUCE-6222 and MAPREDUCE-6394). Let me know what you think when you have time. Thanks. > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch, > MAPREDUCE-6376.003.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6376) Fix long load times of .jhist file in JobHistoryServer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590582#comment-14590582 ] Ray Chiang commented on MAPREDUCE-6376: --- Linking to another JHS speedup JIRA. > Fix long load times of .jhist file in JobHistoryServer > -- > > Key: MAPREDUCE-6376 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6376 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: jobhistoryserver >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > Labels: supportability > Attachments: MAPREDUCE-6376.001.patch, MAPREDUCE-6376.002.patch, > MAPREDUCE-6376.003.patch > > > When you click on a Job link in the JHS Web UI, it loads the .jhist file. > For jobs which have a large number of tasks, the load time can break UI > responsiveness. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590548#comment-14590548
]
Hadoop QA commented on MAPREDUCE-6400:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 0s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 43s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 57s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 3s | The applied patch generated 1
new checkstyle issues (total was 17, now 18). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 38s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 19s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 1m 39s | Tests passed in
hadoop-mapreduce-client-core. |
| {color:red}-1{color} | mapreduce tests | 94m 56s | Tests failed in
hadoop-mapreduce-client-jobclient. |
| | | 137m 25s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.mapred.TestMRTimelineEventHandling |
| | hadoop.ipc.TestMRCJCSocketFactory |
| | hadoop.mapred.TestSpecialCharactersInOutputPath |
| | hadoop.mapred.TestClusterMapReduceTestCase |
| Timed out tests | org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740125/MAPREDUCE-6400.001.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 6e3fcff |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5807/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5807/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-jobclient test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5807/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5807/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5807/console |
This message was automatically generated.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula reassigned MAPREDUCE-6400:
---
Assignee: Brahma Reddy Battula (was: Gera Shegalov)
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated MAPREDUCE-6400:
Attachment: MAPREDUCE-6400-002.patch
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6405) NullPointerException in App Attempts page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590324#comment-14590324 ] Siqi Li commented on MAPREDUCE-6405: Looks like the problem stems from the fact that AttemptsPage extends TaskPage, and the ability of killing attempt is designed for TaskPage only. > NullPointerException in App Attempts page > - > > Key: MAPREDUCE-6405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Siqi Li >Assignee: Siqi Li > Attachments: MAPREDUCE-6405.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-6405) NullPointerException in App Attempts page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li reassigned MAPREDUCE-6405: -- Assignee: Siqi Li > NullPointerException in App Attempts page > - > > Key: MAPREDUCE-6405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Siqi Li >Assignee: Siqi Li > Attachments: MAPREDUCE-6405.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6405) NullPointerException in App Attempts page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li updated MAPREDUCE-6405: --- Status: Patch Available (was: Open) > NullPointerException in App Attempts page > - > > Key: MAPREDUCE-6405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Siqi Li >Assignee: Siqi Li > Attachments: MAPREDUCE-6405.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6405) NullPointerException in App Attempts page
[ https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li updated MAPREDUCE-6405: --- Attachment: MAPREDUCE-6405.v1.patch > NullPointerException in App Attempts page > - > > Key: MAPREDUCE-6405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Siqi Li >Assignee: Siqi Li > Attachments: MAPREDUCE-6405.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590292#comment-14590292
]
Akira AJISAKA commented on MAPREDUCE-6400:
--
bq. A good pattern is when the stream open API consumer closes the stream. In
IFlileInputStream for shuffle, the stream is opened outside, and therefore
should be and is closed outside.
Thanks [~jira.shegalov] for the comment. I agree with you that the pattern is
good. However, before MAPREDUCE-6174, the stream is closed when IOException
occurs. Can we create a separate jira for improving the pattern?
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6405) NullPointerException in App Attempts page
[
https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590250#comment-14590250
]
Siqi Li commented on MAPREDUCE-6405:
Hi [~ryu_kobayashi], It seems that there is a NPE when we access App Attempts
page.
And I think it is coming from this line of code from MAPREDUCE-6364
{code}
String appID = app.getJob().getID().getAppId().toString();
{code}
Can you take a look?
> NullPointerException in App Attempts page
> -
>
> Key: MAPREDUCE-6405
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Siqi Li
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6405) NullPointerException in App Attempts page
[
https://issues.apache.org/jira/browse/MAPREDUCE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590227#comment-14590227
]
Siqi Li commented on MAPREDUCE-6405:
A problem occurs when you access
http://rmhost:rmport/proxy/application_1434432414806_0003/mapreduce/attempts/job_1434432414806_0003/m/RUNNING
{code}
2015-06-15 22:46:56,369 ERROR [378354947@qtp-596129348-3]
org.apache.hadoop.mapreduce.v2.app.webapp.AppController: Failed to render
attempts page with task type : m for job id : job_1434432414806_0003
java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.v2.app.webapp.TaskPage$AttemptsBlock.render(TaskPage.java:74)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
at
org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at
org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:845)
at
org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:56)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212)
at
org.apache.hadoop.mapreduce.v2.app.webapp.AppController.attempts(AppController.java:296)
{code}
> NullPointerException in App Attempts page
> -
>
> Key: MAPREDUCE-6405
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Siqi Li
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6405) NullPointerException in App Attempts page
Siqi Li created MAPREDUCE-6405: -- Summary: NullPointerException in App Attempts page Key: MAPREDUCE-6405 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6405 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.8.0 Reporter: Siqi Li -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590210#comment-14590210
]
Brahma Reddy Battula commented on MAPREDUCE-6400:
-
Thanks a lot [~ajisakaa] , I will address your comments and update the patch
soon..
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590206#comment-14590206
]
Gera Shegalov commented on MAPREDUCE-6400:
--
bq. if inOwner is set to false, we cannot close the wrapped stream when an
exception occurs.
A good pattern is when the stream open API consumer closes the stream. In
IFlileInputStream for shuffle, the stream is opened outside, and therefore
should be and is closed outside.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590174#comment-14590174
]
Akira AJISAKA commented on MAPREDUCE-6400:
--
Thanks [~jira.shegalov] and [~brahmareddy] for creating the patch. In Gera's
patch, if {{inOwner}} is set to false, we cannot close the wrapped stream when
an exception occurs. Therefore I prefer Brahma's one.
Some comments for Brahma's patch.
1. Would you please catch {{IOException}} in {{IFileWrappedMapOutput#shuffle}}?
In the catch clause, we should close the wrapped stream and re-throw the
exception.
2. Would you remove unused imports from TestReduceFetchFromPartialMem.java and
TestFetcher.java?
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6373) The logger reports total input paths but it is referring to input files
[
https://issues.apache.org/jira/browse/MAPREDUCE-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6373:
Attachment: 0002-MAPREDUCE-6373.patch
[~devaraj.k] Have uploaded the patch based on your comment. Also {{import
org.apache.hadoop.mapred.SplitLocationInfo}} unused import have removed as part
of the same.
> The logger reports total input paths but it is referring to input files
> ---
>
> Key: MAPREDUCE-6373
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6373
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Andi Chirita Amdocs
>Assignee: Bibin A Chundatt
>Priority: Trivial
> Labels: newbie
> Attachments: 0002-MAPREDUCE-6373.patch, MAPREDUCE-6373.patch
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> The log message in the FileInputFormat is misleading :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 6
> {code}
> There is only 1 input path and 6 input files so the log message should be :
> {code}
> 2015-04-24 13:12:30,205 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input files to
> process : 6
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6401) Container-launch failure gives no debugging output
[
https://issues.apache.org/jira/browse/MAPREDUCE-6401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589925#comment-14589925
]
Hari Sekhon commented on MAPREDUCE-6401:
Actually the task logs showed the same thing, not much to go on:
{code}
Exception from container-launch. Container id:
container_e199_1434474871820_0001_02_19 Exit code: 7 Stack trace:
ExitCodeException exitCode=7:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:293)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745) Shell output: main : command
provided 1 main : user is main : requested yarn user is
Container exited with a non-zero exit code 7
{code}
but the full tasks logs don't seem to have been retained by the history server.
This made me suspicious so I reset the logging locations to try to get my hands
on the full logs and after a yarn restart jobs started working normally again
without failed tasks/container launches. Although I'm very certain that the
cluster used to log to that dir I reset it to, perhaps Ambari had a bug that
lost the location and reset to debug locations that didn't work properly (it
wouldn't be the first time, eg. AMBARI-9022)
I think we should leave this as a minor todo to improve debugging information,
especially when launching shell commands and encountering non-zero exit codes,
logging is king.
> Container-launch failure gives no debugging output
> --
>
> Key: MAPREDUCE-6401
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6401
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 2.6.0
> Environment: HDP 2.2
>Reporter: Hari Sekhon
> Attachments: job.log
>
>
> MR jobs are failing on my cluster with Stack trace: ExitCodeException
> exitCode=7 but little else in terms of debugging information. Can we please
> improve the debugging info? Log file is attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6401) Container-launch failure gives no debugging output
[ https://issues.apache.org/jira/browse/MAPREDUCE-6401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sekhon updated MAPREDUCE-6401: --- Priority: Minor (was: Major) > Container-launch failure gives no debugging output > -- > > Key: MAPREDUCE-6401 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6401 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.6.0 > Environment: HDP 2.2 >Reporter: Hari Sekhon >Priority: Minor > Attachments: job.log > > > MR jobs are failing on my cluster with Stack trace: ExitCodeException > exitCode=7 but little else in terms of debugging information. Can we please > improve the debugging info? Log file is attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589875#comment-14589875
]
Gera Shegalov commented on MAPREDUCE-6400:
--
As a pattern, having close looks better to me, although in this case it ends
doing nothing because there is no readahead and I disabled the wrapped stream
close . [~eepayne], [~jlowe], do you have a preference?
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated MAPREDUCE-6400:
-
Attachment: MAPREDUCE-6400.001.patch
Uploading my patch for completeness. Don't mind if we go with [~brahmareddy]'s,
either.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589862#comment-14589862
]
Brahma Reddy Battula commented on MAPREDUCE-6400:
-
Will give patch to fix the checkstyle..
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated MAPREDUCE-6400:
-
Attachment: (was: MAPREDUCE-6400.001.patch)
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-3384) Add warning message for underflow or overflow
[
https://issues.apache.org/jira/browse/MAPREDUCE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589802#comment-14589802
]
Brahma Reddy Battula commented on MAPREDUCE-3384:
-
Attached the patch based on above logic..Kindly review..
> Add warning message for underflow or overflow
> -
>
> Key: MAPREDUCE-3384
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3384
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: JiangKai
>Assignee: Brahma Reddy Battula
> Attachments: MAPREDUCE-3384-002.patch, MAPREDUCE-3384-003.patch,
> MAPREDUCE-3384-004.patch, MAPREDUCE-3384.patch
>
>
> When we call the function reduce() of LongSumReducer,the result may overflow
> or underflow.
> We should have a warning message to users if overflow/underflow occurs for
> all these classes,
> {code:xml}
> /java/org/apache/hadoop/mapred/lib/LongSumReducer.java
> .../apache/hadoop/mapreduce/lib/reduce/IntSumReducer.java
> .../apache/hadoop/mapreduce/lib/reduce/LongSumReducer.java
> .../test/java/org/apache/hadoop/fs/AccumulatingReducer.java
> .../hadoop/mapred/nativetask/kvtest/HashSumReducer.java
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-3384) Add warning message for underflow or overflow
[
https://issues.apache.org/jira/browse/MAPREDUCE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated MAPREDUCE-3384:
Status: Patch Available (was: Open)
> Add warning message for underflow or overflow
> -
>
> Key: MAPREDUCE-3384
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3384
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: JiangKai
>Assignee: Brahma Reddy Battula
> Attachments: MAPREDUCE-3384-002.patch, MAPREDUCE-3384-003.patch,
> MAPREDUCE-3384-004.patch, MAPREDUCE-3384.patch
>
>
> When we call the function reduce() of LongSumReducer,the result may overflow
> or underflow.
> We should have a warning message to users if overflow/underflow occurs for
> all these classes,
> {code:xml}
> /java/org/apache/hadoop/mapred/lib/LongSumReducer.java
> .../apache/hadoop/mapreduce/lib/reduce/IntSumReducer.java
> .../apache/hadoop/mapreduce/lib/reduce/LongSumReducer.java
> .../test/java/org/apache/hadoop/fs/AccumulatingReducer.java
> .../hadoop/mapred/nativetask/kvtest/HashSumReducer.java
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-3384) Add warning message for underflow or overflow
[
https://issues.apache.org/jira/browse/MAPREDUCE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated MAPREDUCE-3384:
Attachment: MAPREDUCE-3384-004.patch
> Add warning message for underflow or overflow
> -
>
> Key: MAPREDUCE-3384
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3384
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: JiangKai
>Assignee: Brahma Reddy Battula
> Attachments: MAPREDUCE-3384-002.patch, MAPREDUCE-3384-003.patch,
> MAPREDUCE-3384-004.patch, MAPREDUCE-3384.patch
>
>
> When we call the function reduce() of LongSumReducer,the result may overflow
> or underflow.
> We should have a warning message to users if overflow/underflow occurs for
> all these classes,
> {code:xml}
> /java/org/apache/hadoop/mapred/lib/LongSumReducer.java
> .../apache/hadoop/mapreduce/lib/reduce/IntSumReducer.java
> .../apache/hadoop/mapreduce/lib/reduce/LongSumReducer.java
> .../test/java/org/apache/hadoop/fs/AccumulatingReducer.java
> .../hadoop/mapred/nativetask/kvtest/HashSumReducer.java
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589780#comment-14589780
]
Hadoop QA commented on MAPREDUCE-6400:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 4s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 37s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 46s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 46s | The applied patch generated 1
new checkstyle issues (total was 0, now 1). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 26s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 1m 37s | Tests passed in
hadoop-mapreduce-client-core. |
| | | 39m 49s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740064/MAPREDUCE-6400.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / ebb9a82 |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5806/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5806/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5806/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5806/console |
This message was automatically generated.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6404) Allow AM to specify a port range for starting its webapp
Varun Saxena created MAPREDUCE-6404: --- Summary: Allow AM to specify a port range for starting its webapp Key: MAPREDUCE-6404 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6404 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Varun Saxena Assignee: Varun Saxena Allow AM to specify a port range for starting its webapp -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-6402) Application Master listen on all IPs
[ https://issues.apache.org/jira/browse/MAPREDUCE-6402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena reassigned MAPREDUCE-6402: --- Assignee: Varun Saxena > Application Master listen on all IPs > > > Key: MAPREDUCE-6402 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6402 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: applicationmaster >Affects Versions: 2.7.0 >Reporter: Peter Shi >Assignee: Varun Saxena > > The application master listening on all IPs, which may cause security > problems. Similar issue: MAPREDUCE-5938 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6402) Application Master listen on all IPs
[ https://issues.apache.org/jira/browse/MAPREDUCE-6402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589594#comment-14589594 ] nijel commented on MAPREDUCE-6402: -- Hi, One query How the AM IP can be configured ? User/job client wont be able to decide which NM it launches. > Application Master listen on all IPs > > > Key: MAPREDUCE-6402 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6402 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: applicationmaster >Affects Versions: 2.7.0 >Reporter: Peter Shi > > The application master listening on all IPs, which may cause security > problems. Similar issue: MAPREDUCE-5938 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6402) Application Master listen on all IPs
[ https://issues.apache.org/jira/browse/MAPREDUCE-6402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated MAPREDUCE-6402: Assignee: (was: Varun Saxena) > Application Master listen on all IPs > > > Key: MAPREDUCE-6402 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6402 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: applicationmaster >Affects Versions: 2.7.0 >Reporter: Peter Shi > > The application master listening on all IPs, which may cause security > problems. Similar issue: MAPREDUCE-5938 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-6402) Application Master listen on all IPs
[ https://issues.apache.org/jira/browse/MAPREDUCE-6402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena reassigned MAPREDUCE-6402: --- Assignee: Varun Saxena > Application Master listen on all IPs > > > Key: MAPREDUCE-6402 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6402 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: applicationmaster >Affects Versions: 2.7.0 >Reporter: Peter Shi >Assignee: Varun Saxena > > The application master listening on all IPs, which may cause security > problems. Similar issue: MAPREDUCE-5938 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6401) Container-launch failure gives no debugging output
[ https://issues.apache.org/jira/browse/MAPREDUCE-6401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589538#comment-14589538 ] Devaraj K commented on MAPREDUCE-6401: -- The attached log seems to be MRAppMaster log(not the task logs), where you could see overall MR Job(all the tasks) logs. bq. I'm trying to fix it blindfolded. I understand that these logs can be improved further to get the detailed failed information instead of just exit codes. If you want to know the reason for container launch failure, you can check the corresponding failed container/task log by going to Job History Server UI(if log aggregation enabled) or directly by going to the app log dir’s for the container stderr/stdout in the failed node. > Container-launch failure gives no debugging output > -- > > Key: MAPREDUCE-6401 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6401 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.6.0 > Environment: HDP 2.2 >Reporter: Hari Sekhon > Attachments: job.log > > > MR jobs are failing on my cluster with Stack trace: ExitCodeException > exitCode=7 but little else in terms of debugging information. Can we please > improve the debugging info? Log file is attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6401) Container-launch failure gives no debugging output
[ https://issues.apache.org/jira/browse/MAPREDUCE-6401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589509#comment-14589509 ] Hari Sekhon commented on MAPREDUCE-6401: Yes the attached log is from the container task logs obtained via the RM. I would have expected a failing shell call should print the command, status code and both stderr and stdout with which to debug/reproduce. It's clear something is different since these are new nodes in the cluster that are giving these problems but it's not clear what is wrong because the debug output is so vague - I'm trying to fix it blindfolded. > Container-launch failure gives no debugging output > -- > > Key: MAPREDUCE-6401 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6401 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.6.0 > Environment: HDP 2.2 >Reporter: Hari Sekhon > Attachments: job.log > > > MR jobs are failing on my cluster with Stack trace: ExitCodeException > exitCode=7 but little else in terms of debugging information. Can we please > improve the debugging info? Log file is attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589440#comment-14589440
]
Brahma Reddy Battula commented on MAPREDUCE-6400:
-
[~jira.shegalov] do you mind,If I give patch..?
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated MAPREDUCE-6400:
Attachment: MAPREDUCE-6400.patch
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589417#comment-14589417
]
Akira AJISAKA commented on MAPREDUCE-6400:
--
Thanks [~jira.shegalov] for taking this issue. The uploaded patch looks to be
for HDFS-8591. Would you upload a patch for this issue?
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated MAPREDUCE-6400:
-
Component/s: (was: test)
task
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated MAPREDUCE-6400:
-
Status: Patch Available (was: Open)
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
> Attachments: MAPREDUCE-6400.001.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated MAPREDUCE-6400:
-
Attachment: MAPREDUCE-6400.001.patch
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
> Attachments: MAPREDUCE-6400.001.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated MAPREDUCE-6400:
-
Priority: Blocker (was: Major)
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589400#comment-14589400
]
Gera Shegalov commented on MAPREDUCE-6400:
--
In MAPREDUCE-6174, [~eepayne] made sure that IFileInputStream is closed after
shuffle. This was not the case before, but I do think it's a good idea. But we
should not close the wrapped physical stream because it shared for all outputs
coming from the same node.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov reassigned MAPREDUCE-6400:
Assignee: Gera Shegalov (was: Brahma Reddy Battula)
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
