[jira] [Commented] (MAPREDUCE-6580) Test failure : TestMRJobsWithProfiler
[
https://issues.apache.org/jira/browse/MAPREDUCE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207986#comment-15207986
]
Hudson commented on MAPREDUCE-6580:
---
FAILURE: Integrated in Hadoop-trunk-Commit #9487 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/9487/])
MAPREDUCE-6580. Test failure: TestMRJobsWithProfiler. Contributed by (aajisaka:
rev 0bfe5a0167f0af65f1d7a8bc251a1c404ed41316)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobsWithProfiler.java
> Test failure : TestMRJobsWithProfiler
> -
>
> Key: MAPREDUCE-6580
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6580
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Eric Badger
> Fix For: 2.8.0, 2.7.3
>
> Attachments: MAPREDUCE-6580.001.patch
>
>
> From
> [https://builds.apache.org/job/PreCommit-YARN-Build/9976/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt]
> TestMRJobsWithProfiler fails intermittently
> {code}
> Running org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 212.973 sec
> <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> testDifferentProfilers(org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler)
> Time elapsed: 133.116 sec <<< FAILURE!
> java.lang.AssertionError: expected:<4> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testProfilerInternal(TestMRJobsWithProfiler.java:212)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testDifferentProfilers(TestMRJobsWithProfiler.java:117)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6580) Test failure : TestMRJobsWithProfiler
[
https://issues.apache.org/jira/browse/MAPREDUCE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15206814#comment-15206814
]
Akira AJISAKA commented on MAPREDUCE-6580:
--
Nice catch, Eric! LGTM, +1.
> Test failure : TestMRJobsWithProfiler
> -
>
> Key: MAPREDUCE-6580
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6580
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Eric Badger
> Attachments: MAPREDUCE-6580.001.patch
>
>
> From
> [https://builds.apache.org/job/PreCommit-YARN-Build/9976/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt]
> TestMRJobsWithProfiler fails intermittently
> {code}
> Running org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 212.973 sec
> <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> testDifferentProfilers(org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler)
> Time elapsed: 133.116 sec <<< FAILURE!
> java.lang.AssertionError: expected:<4> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testProfilerInternal(TestMRJobsWithProfiler.java:212)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testDifferentProfilers(TestMRJobsWithProfiler.java:117)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6580) Test failure : TestMRJobsWithProfiler
[
https://issues.apache.org/jira/browse/MAPREDUCE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15206503#comment-15206503
]
Eric Badger commented on MAPREDUCE-6580:
TestUberAM and TestMRJobs both have failed recent builds in [MAPREDUCE-6315]
and are unrelated to this patch.
TestMRIntermediateDataEncryption fails intermittently with a timeout in recent
builds [MAPREDUCE-6579], [YARN-4340].
> Test failure : TestMRJobsWithProfiler
> -
>
> Key: MAPREDUCE-6580
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6580
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Eric Badger
> Attachments: MAPREDUCE-6580.001.patch
>
>
> From
> [https://builds.apache.org/job/PreCommit-YARN-Build/9976/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt]
> TestMRJobsWithProfiler fails intermittently
> {code}
> Running org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 212.973 sec
> <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> testDifferentProfilers(org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler)
> Time elapsed: 133.116 sec <<< FAILURE!
> java.lang.AssertionError: expected:<4> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testProfilerInternal(TestMRJobsWithProfiler.java:212)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testDifferentProfilers(TestMRJobsWithProfiler.java:117)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6580) Test failure : TestMRJobsWithProfiler
[
https://issues.apache.org/jira/browse/MAPREDUCE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205933#comment-15205933
]
Hadoop QA commented on MAPREDUCE-6580:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
56s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 28s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 51s {color}
| {color:red} hadoop-mapreduce-client-jobclient in the patch failed with JDK
v1.8.0_74. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 110m 49s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 28s
{color} | {color:red} Patch generated 19 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 219m 22s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_74 Failed junit tests | hadoop.mapreduce.v2.TestUberAM |
| | hadoop.mapreduce.v2.TestMRJobs |
| JDK v1.7.0_95 Failed junit tests | hadoop.mapreduce.v2.TestUberAM |
| | hadoop.mapreduce.v2.TestMRJobs |
| JDK v1.7.0_95 Timed out junit tests |
org.apache.hadoop.mapred.TestMRIntermediateDataEncryption |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:fbe3e86 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12794629/MAPREDUCE-6580.001.patch
|
| JIRA Issue | MAPREDUCE-6580 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs
[jira] [Commented] (MAPREDUCE-6580) Test failure : TestMRJobsWithProfiler
[
https://issues.apache.org/jira/browse/MAPREDUCE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205113#comment-15205113
]
Eric Badger commented on MAPREDUCE-6580:
I am able to get TestMRJobsWithProfiler#testDifferentProfilers to fail
consistently on my Mac. However, I am not able to get it to fail on my Linux
box or on either box when running testDefaultProfiler. On my mac, I have found
that the problem is with the hprof option "times" in
TestMRJobsWithProfiler.java:137
(-agentlib:hprof=cpu=times,heap=sites,force=n,thread=y,verbose=n," + "file=%s).
This is causing the JVM to screw up and fail seemingly before it even begins to
start the container.
Additionally, the "times" option is incredibly expensive and slows the JVM down
a ton, making the test very likely to timeout on a busy Jenkins instance given
the 150 second test timeout. IMO using "samples" instead of "times" would be
much better. Using samples, the test passes on my mac and speeds the test up by
almost 2x (74.8s vs. 145.5s on my Linux machine). Using "samples" would get rid
of the hprof error on my mac and also speed up the test considerably, reducing
the chance of test timeouts.
Is there any reason that we can't use "samples" instead of "times" for the
profiler?
> Test failure : TestMRJobsWithProfiler
> -
>
> Key: MAPREDUCE-6580
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6580
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>
> From
> [https://builds.apache.org/job/PreCommit-YARN-Build/9976/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt]
> TestMRJobsWithProfiler fails intermittently
> {code}
> Running org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 212.973 sec
> <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> testDifferentProfilers(org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler)
> Time elapsed: 133.116 sec <<< FAILURE!
> java.lang.AssertionError: expected:<4> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testProfilerInternal(TestMRJobsWithProfiler.java:212)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testDifferentProfilers(TestMRJobsWithProfiler.java:117)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
