[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599566#comment-14599566
]
Jason Lowe commented on MAPREDUCE-6400:
---
I suspect that connection.disconnect() is already closing the underlying input
stream, but it shouldn't hurt if we close it early just in case it doesn't.
+1 for the latest patch. Committing this.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.003.patch, MAPREDUCE-6400.004.patch, MAPREDUCE-6400.005.patch,
> MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599104#comment-14599104
]
Akira AJISAKA commented on MAPREDUCE-6400:
--
+1, the latest simple patch looks good to me. Thanks [~jlowe] for the comment
and [~jira.shegalov] for updating the patch.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.003.patch, MAPREDUCE-6400.004.patch, MAPREDUCE-6400.005.patch,
> MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598841#comment-14598841
]
Hadoop QA commented on MAPREDUCE-6400:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 42s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 52s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 53s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 50s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 41s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 27s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 1m 47s | Tests passed in
hadoop-mapreduce-client-core. |
| | | 41m 9s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12741435/MAPREDUCE-6400.005.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 49dfad9 |
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5835/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5835/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5835/console |
This message was automatically generated.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.003.patch, MAPREDUCE-6400.004.patch, MAPREDUCE-6400.005.patch,
> MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596349#comment-14596349
]
Gera Shegalov commented on MAPREDUCE-6400:
--
bq. We don't want to spend time reading the rest of the data, rather we just
want to abandon the transfer and move on.
Good point that we don't care about checksum verification. Initially I was only
worried about the read-ahead pool but it's not used for the shuffled data
arriving via a TCP socket.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.003.patch, MAPREDUCE-6400.004.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14595999#comment-14595999
]
Jason Lowe commented on MAPREDUCE-6400:
---
bq. do you mean iFin in the 003 patch should be changed to the wrapped stream?
Actually no, I'm not a big fan of that approach either. Again, it seems weird
to me that we're closing the stream only sometimes, and that sometimes is only
when something fails. IMHO the code that is fielding the error and calling
abort should be the one closing the stream, as this is likely the one that
passed the stream into the map output in the first place. As Gera mentioned
above, it's simplest if the code that opens the stream closes the stream. It
gets confusing when the ownership is conditional like this.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.003.patch, MAPREDUCE-6400.004.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14595825#comment-14595825
]
Hadoop QA commented on MAPREDUCE-6400:
--
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 32s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 1s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 30s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 31s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 19s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 14s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 1m 36s | Tests passed in
hadoop-mapreduce-client-core. |
| {color:green}+1{color} | mapreduce tests | 109m 1s | Tests passed in
hadoop-mapreduce-client-jobclient. |
| | | 150m 17s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740987/MAPREDUCE-6400.004.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 445b132 |
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5822/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-jobclient test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5822/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5822/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5822/console |
This message was automatically generated.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.003.patch, MAPREDUCE-6400.004.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14595537#comment-14595537
]
Akira AJISAKA commented on MAPREDUCE-6400:
--
Hi [~jlowe], do you mean {{iFin}} in the 003 patch should be changed to the
wrapped stream?
{code:title=IFileWrappedMapOutput.java}
@Override
public void abort() {
IOUtils.cleanup(LOG, iFin);
}
{code}
I'm +1 for doing this.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.003.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14592637#comment-14592637
]
Jason Lowe commented on MAPREDUCE-6400:
---
Sorry for arriving late. I'm a bit confused on the latest patch. We're going
to close the IFileInputStream but only when something fails? That seems odd.
In addition, note that IFileInputStream.close is not trivial -- it will consume
the rest of the stream to verify the checksum, but we don't care since we're
only trying to close when something else went wrong. At that point the
checksum being valid is not important, correct? I'm thinking of a scenario
where writing to the local disk fails early in a very large transfer. We don't
want to spend time reading the rest of the data, rather we just want to abandon
the transfer and move on.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.003.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14592290#comment-14592290
]
Hadoop QA commented on MAPREDUCE-6400:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 13s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 42s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 52s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 21s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 16s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 1m 41s | Tests passed in
hadoop-mapreduce-client-core. |
| {color:red}-1{color} | mapreduce tests | 64m 31s | Tests failed in
hadoop-mapreduce-client-jobclient. |
| | | 107m 13s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.mapred.TestMiniMRClientCluster |
| Timed out tests |
org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740336/MAPREDUCE-6400.003.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 1c13519 |
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5814/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-jobclient test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5814/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5814/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5814/console |
This message was automatically generated.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.003.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591172#comment-14591172
]
Hadoop QA commented on MAPREDUCE-6400:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 5s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 41s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 56s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 0s | The applied patch generated 1
new checkstyle issues (total was 0, now 1). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 31s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 15s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | mapreduce tests | 1m 33s | Tests failed in
hadoop-mapreduce-client-core. |
| {color:red}-1{color} | mapreduce tests | 105m 34s | Tests failed in
hadoop-mapreduce-client-jobclient. |
| | | 147m 53s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.mapred.TestLineRecordReader |
| | hadoop.mapreduce.lib.input.TestLineRecordReader |
| | hadoop.mapred.TestMiniMRChildTask |
| | hadoop.mapred.TestMerge |
| | hadoop.mapred.TestReduceFetch |
| | hadoop.mapreduce.v2.TestUberAM |
| | hadoop.mapred.TestReduceFetchFromPartialMem |
| | hadoop.mapred.TestMRIntermediateDataEncryption |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740193/MAPREDUCE-6400-002.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / cc43288 |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5810/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5810/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-jobclient test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5810/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5810/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5810/console |
This message was automatically generated.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590548#comment-14590548
]
Hadoop QA commented on MAPREDUCE-6400:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 0s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 43s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 57s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 3s | The applied patch generated 1
new checkstyle issues (total was 17, now 18). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 38s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 19s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 1m 39s | Tests passed in
hadoop-mapreduce-client-core. |
| {color:red}-1{color} | mapreduce tests | 94m 56s | Tests failed in
hadoop-mapreduce-client-jobclient. |
| | | 137m 25s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.mapred.TestMRTimelineEventHandling |
| | hadoop.ipc.TestMRCJCSocketFactory |
| | hadoop.mapred.TestSpecialCharactersInOutputPath |
| | hadoop.mapred.TestClusterMapReduceTestCase |
| Timed out tests | org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740125/MAPREDUCE-6400.001.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 6e3fcff |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5807/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5807/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-jobclient test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5807/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5807/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5807/console |
This message was automatically generated.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Attachments: MAPREDUCE-6400-002.patch, MAPREDUCE-6400.001.patch,
> MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590292#comment-14590292
]
Akira AJISAKA commented on MAPREDUCE-6400:
--
bq. A good pattern is when the stream open API consumer closes the stream. In
IFlileInputStream for shuffle, the stream is opened outside, and therefore
should be and is closed outside.
Thanks [~jira.shegalov] for the comment. I agree with you that the pattern is
good. However, before MAPREDUCE-6174, the stream is closed when IOException
occurs. Can we create a separate jira for improving the pattern?
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590210#comment-14590210
]
Brahma Reddy Battula commented on MAPREDUCE-6400:
-
Thanks a lot [~ajisakaa] , I will address your comments and update the patch
soon..
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590206#comment-14590206
]
Gera Shegalov commented on MAPREDUCE-6400:
--
bq. if inOwner is set to false, we cannot close the wrapped stream when an
exception occurs.
A good pattern is when the stream open API consumer closes the stream. In
IFlileInputStream for shuffle, the stream is opened outside, and therefore
should be and is closed outside.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590174#comment-14590174
]
Akira AJISAKA commented on MAPREDUCE-6400:
--
Thanks [~jira.shegalov] and [~brahmareddy] for creating the patch. In Gera's
patch, if {{inOwner}} is set to false, we cannot close the wrapped stream when
an exception occurs. Therefore I prefer Brahma's one.
Some comments for Brahma's patch.
1. Would you please catch {{IOException}} in {{IFileWrappedMapOutput#shuffle}}?
In the catch clause, we should close the wrapped stream and re-throw the
exception.
2. Would you remove unused imports from TestReduceFetchFromPartialMem.java and
TestFetcher.java?
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589875#comment-14589875
]
Gera Shegalov commented on MAPREDUCE-6400:
--
As a pattern, having close looks better to me, although in this case it ends
doing nothing because there is no readahead and I disabled the wrapped stream
close . [~eepayne], [~jlowe], do you have a preference?
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589862#comment-14589862
]
Brahma Reddy Battula commented on MAPREDUCE-6400:
-
Will give patch to fix the checkstyle..
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589780#comment-14589780
]
Hadoop QA commented on MAPREDUCE-6400:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 4s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 37s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 46s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 46s | The applied patch generated 1
new checkstyle issues (total was 0, now 1). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 26s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 1m 37s | Tests passed in
hadoop-mapreduce-client-core. |
| | | 39m 49s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740064/MAPREDUCE-6400.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / ebb9a82 |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5806/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5806/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5806/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5806/console |
This message was automatically generated.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch, MAPREDUCE-6400.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589440#comment-14589440
]
Brahma Reddy Battula commented on MAPREDUCE-6400:
-
[~jira.shegalov] do you mind,If I give patch..?
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589417#comment-14589417
]
Akira AJISAKA commented on MAPREDUCE-6400:
--
Thanks [~jira.shegalov] for taking this issue. The uploaded patch looks to be
for HDFS-8591. Would you upload a patch for this issue?
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: MAPREDUCE-6400.001.patch
>
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589400#comment-14589400
]
Gera Shegalov commented on MAPREDUCE-6400:
--
In MAPREDUCE-6174, [~eepayne] made sure that IFileInputStream is closed after
shuffle. This was not the case before, but I do think it's a good idea. But we
should not close the wrapped physical stream because it shared for all outputs
coming from the same node.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588557#comment-14588557
]
Gera Shegalov commented on MAPREDUCE-6400:
--
Thanks for pinging [~ajisakaa]. Taking a look!
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>Assignee: Brahma Reddy Battula
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6400) TestReduceFetchFromPartialMem fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587218#comment-14587218
]
Akira AJISAKA commented on MAPREDUCE-6400:
--
I used {{git bisect}} and found this issue was broken by MAPREDUCE-6174.
> TestReduceFetchFromPartialMem fails
> ---
>
> Key: MAPREDUCE-6400
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6400
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Akira AJISAKA
>
> TestReduceFetchFromPartialMem fails.
> {noformat}
> Running org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 132.978 sec
> <<< FAILURE! - in org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
> testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem)
> Time elapsed: 69.214 sec <<< ERROR!
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:867)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.runJob(TestReduceFetchFromPartialMem.java:300)
> at
> org.apache.hadoop.mapred.TestReduceFetchFromPartialMem.testReduceFromPartialMem(TestReduceFetchFromPartialMem.java:93)
> Results :
> Tests in error:
> TestReduceFetchFromPartialMem.testReduceFromPartialMem:93->runJob:300 » IO
> Job...
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
