[jira] [Commented] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802573#comment-13802573 ] Jonathan Eagles commented on MAPREDUCE-4980: Sorry. Those seems to with the build itself and not due to this patch. > Parallel test execution of hadoop-mapreduce-client-core > --- > > Key: MAPREDUCE-4980 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 3.0.0 >Reporter: Tsuyoshi OZAWA >Assignee: Andrey Klochkov > Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch, > MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch, > MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch > > > The maven surefire plugin supports parallel testing feature. By using it, the > tests can be run more faster. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5221) Reduce side Combiner is not used when using the new API
[
https://issues.apache.org/jira/browse/MAPREDUCE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802564#comment-13802564
]
Hadoop QA commented on MAPREDUCE-5221:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609792/MAPREDUCE-5221.7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4132//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4132//console
This message is automatically generated.
> Reduce side Combiner is not used when using the new API
> ---
>
> Key: MAPREDUCE-5221
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5221
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.0.4-alpha
>Reporter: Siddharth Seth
>Assignee: Tsuyoshi OZAWA
> Attachments: MAPREDUCE-5221.1.patch, MAPREDUCE-5221.2.patch,
> MAPREDUCE-5221.3.patch, MAPREDUCE-5221.4.patch, MAPREDUCE-5221.5.patch,
> MAPREDUCE-5221.6.patch, MAPREDUCE-5221.7.patch
>
>
> If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this
> will silently ignored on the reduce side since the reduce side usage is only
> aware of the old api combiner.
> This doesn't fail the job - since the new combiner key does not deprecate the
> old key.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (MAPREDUCE-5221) Reduce side Combiner is not used when using the new API
[ https://issues.apache.org/jira/browse/MAPREDUCE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-5221: -- Attachment: MAPREDUCE-5221.7.patch > Reduce side Combiner is not used when using the new API > --- > > Key: MAPREDUCE-5221 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5221 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.0.4-alpha >Reporter: Siddharth Seth >Assignee: Tsuyoshi OZAWA > Attachments: MAPREDUCE-5221.1.patch, MAPREDUCE-5221.2.patch, > MAPREDUCE-5221.3.patch, MAPREDUCE-5221.4.patch, MAPREDUCE-5221.5.patch, > MAPREDUCE-5221.6.patch, MAPREDUCE-5221.7.patch > > > If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this > will silently ignored on the reduce side since the reduce side usage is only > aware of the old api combiner. > This doesn't fail the job - since the new combiner key does not deprecate the > old key. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[
https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802553#comment-13802553
]
Jonathan Eagles commented on MAPREDUCE-4980:
Looks like many failures like this in the unit tests runs as show in the
console log
{quote}
---
T E S T S
---
Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread
{quote}
Also, this
{quote}
[WARNING] The requested profile "native" could not be activated because it does
not exist.
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on
project hadoop-mapreduce-client-jobclient: ExecutionException; nested exception
is java.util.concurrent.ExecutionException: java.lang.RuntimeException: The
forked VM terminated without saying properly goodbye. VM crash or System.exit
called ?
{quote}
> Parallel test execution of hadoop-mapreduce-client-core
> ---
>
> Key: MAPREDUCE-4980
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980
> Project: Hadoop Map/Reduce
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0
>Reporter: Tsuyoshi OZAWA
>Assignee: Andrey Klochkov
> Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch,
> MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch,
> MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch
>
>
> The maven surefire plugin supports parallel testing feature. By using it, the
> tests can be run more faster.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-4282) Convert Forrest docs to APT
[
https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802457#comment-13802457
]
Hadoop QA commented on MAPREDUCE-4282:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609756/MAPREDUCE-4282-tutorial.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4131//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4131//console
This message is automatically generated.
> Convert Forrest docs to APT
> ---
>
> Key: MAPREDUCE-4282
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282
> Project: Hadoop Map/Reduce
> Issue Type: Task
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Akira AJISAKA
> Labels: newbie
> Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch,
> MAPREDUCE-4282-2.patch, MAPREDUCE-4282-tutorial.patch, MapReduceTutorial.html
>
>
> MR side of HADOOP-8427. Not all of the old forrest docs in
> src/documentation/content/xdocs have been converted over to APT yet, let's do
> that and remove the forrest docs.
>
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-4282: - Target Version/s: 2.3.0 Status: Patch Available (was: Open) > Convert Forrest docs to APT > --- > > Key: MAPREDUCE-4282 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 > Project: Hadoop Map/Reduce > Issue Type: Task >Affects Versions: 2.0.0-alpha >Reporter: Eli Collins >Assignee: Akira AJISAKA > Labels: newbie > Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, > MAPREDUCE-4282-2.patch, MAPREDUCE-4282-tutorial.patch, MapReduceTutorial.html > > > MR side of HADOOP-8427. Not all of the old forrest docs in > src/documentation/content/xdocs have been converted over to APT yet, let's do > that and remove the forrest docs. > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-4282: - Attachment: MAPREDUCE-4282-tutorial.patch > Convert Forrest docs to APT > --- > > Key: MAPREDUCE-4282 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 > Project: Hadoop Map/Reduce > Issue Type: Task >Affects Versions: 2.0.0-alpha >Reporter: Eli Collins >Assignee: Akira AJISAKA > Labels: newbie > Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, > MAPREDUCE-4282-2.patch, MAPREDUCE-4282-tutorial.patch, MapReduceTutorial.html > > > MR side of HADOOP-8427. Not all of the old forrest docs in > src/documentation/content/xdocs have been converted over to APT yet, let's do > that and remove the forrest docs. > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-4282: - Attachment: MapReduceTutorial.html > Convert Forrest docs to APT > --- > > Key: MAPREDUCE-4282 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 > Project: Hadoop Map/Reduce > Issue Type: Task >Affects Versions: 2.0.0-alpha >Reporter: Eli Collins >Assignee: Akira AJISAKA > Labels: newbie > Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, > MAPREDUCE-4282-2.patch, MapReduceTutorial.html > > > MR side of HADOOP-8427. Not all of the old forrest docs in > src/documentation/content/xdocs have been converted over to APT yet, let's do > that and remove the forrest docs. > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-4282: - Target Version/s: (was: ) Status: Open (was: Patch Available) > Convert Forrest docs to APT > --- > > Key: MAPREDUCE-4282 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 > Project: Hadoop Map/Reduce > Issue Type: Task >Affects Versions: 2.0.0-alpha >Reporter: Eli Collins >Assignee: Akira AJISAKA > Labels: newbie > Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, > MAPREDUCE-4282-2.patch, MapReduceTutorial.html > > > MR side of HADOOP-8427. Not all of the old forrest docs in > src/documentation/content/xdocs have been converted over to APT yet, let's do > that and remove the forrest docs. > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802435#comment-13802435 ] Akira AJISAKA commented on MAPREDUCE-4282: -- I converted MapReduce Tutorial and attaching a patch. The patch is not only to convert forrest to APT but also to * Update MRv1-specific word (ex. jobtracker, tasktracker) to MRv2(YARN). * Use new API and properties. I also attach a html. > Convert Forrest docs to APT > --- > > Key: MAPREDUCE-4282 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 > Project: Hadoop Map/Reduce > Issue Type: Task >Affects Versions: 2.0.0-alpha >Reporter: Eli Collins >Assignee: Akira AJISAKA > Labels: newbie > Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, > MAPREDUCE-4282-2.patch, MapReduceTutorial.html > > > MR side of HADOOP-8427. Not all of the old forrest docs in > src/documentation/content/xdocs have been converted over to APT yet, let's do > that and remove the forrest docs. > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA reassigned MAPREDUCE-4282: Assignee: Akira AJISAKA (was: Eli Reisman) > Convert Forrest docs to APT > --- > > Key: MAPREDUCE-4282 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 > Project: Hadoop Map/Reduce > Issue Type: Task >Affects Versions: 2.0.0-alpha >Reporter: Eli Collins >Assignee: Akira AJISAKA > Labels: newbie > Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, > MAPREDUCE-4282-2.patch > > > MR side of HADOOP-8427. Not all of the old forrest docs in > src/documentation/content/xdocs have been converted over to APT yet, let's do > that and remove the forrest docs. > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5186) mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802414#comment-13802414 ] Karthik Kambatla commented on MAPREDUCE-5186: - We recently ran into this as well. While the check is superficial (throws a warning) in Hadoop-1, it throws an Exception essentially failing the job submission in Hadoop-2. MAPREDUCE-4146 seems to have introduced this. [~tomwhite], do you remember the reason for throwing an Exception. If there is no particular reason, it would be nice to revert to old behavior. > mapreduce.job.max.split.locations causes some splits created by > CombineFileInputFormat to fail > -- > > Key: MAPREDUCE-5186 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5186 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission >Affects Versions: 2.0.4-alpha, 2.2.0 >Reporter: Sangjin Lee >Priority: Critical > > CombineFileInputFormat can easily create splits that can come from many > different locations (during the last pass of creating "global" splits). > However, we observe that this often runs afoul of the > mapreduce.job.max.split.locations check that's done by JobSplitWriter. > The default value for mapreduce.job.max.split.locations is 10, and with any > decent size cluster, CombineFileInputFormat creates splits that are well > above this limit. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5186) mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-5186: Component/s: (was: mrv1) (was: mrv2) job submission > mapreduce.job.max.split.locations causes some splits created by > CombineFileInputFormat to fail > -- > > Key: MAPREDUCE-5186 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5186 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission >Affects Versions: 2.0.4-alpha, 2.2.0 >Reporter: Sangjin Lee >Priority: Critical > > CombineFileInputFormat can easily create splits that can come from many > different locations (during the last pass of creating "global" splits). > However, we observe that this often runs afoul of the > mapreduce.job.max.split.locations check that's done by JobSplitWriter. > The default value for mapreduce.job.max.split.locations is 10, and with any > decent size cluster, CombineFileInputFormat creates splits that are well > above this limit. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5186) mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-5186: Affects Version/s: 2.2.0 > mapreduce.job.max.split.locations causes some splits created by > CombineFileInputFormat to fail > -- > > Key: MAPREDUCE-5186 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5186 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 2.0.4-alpha, 2.2.0 >Reporter: Sangjin Lee >Priority: Critical > > CombineFileInputFormat can easily create splits that can come from many > different locations (during the last pass of creating "global" splits). > However, we observe that this often runs afoul of the > mapreduce.job.max.split.locations check that's done by JobSplitWriter. > The default value for mapreduce.job.max.split.locations is 10, and with any > decent size cluster, CombineFileInputFormat creates splits that are well > above this limit. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802390#comment-13802390 ] Hudson commented on MAPREDUCE-5561: --- SUCCESS: Integrated in Hadoop-trunk-Commit #4645 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/4645/]) MAPREDUCE-5561. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk. Contributed by Karthik Kambatla (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1534832) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on > trunk > - > > Key: MAPREDUCE-5561 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Cindy Li >Assignee: Karthik Kambatla >Priority: Critical > Fix For: 3.0.0, 2.3.0, 0.23.10, 2.2.1 > > Attachments: mr-5561-1.patch, > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt > > > Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec > <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) > Time elapsed: 5.507 sec <<< FAILURE! > java.lang.AssertionError: expected: but was: > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:147) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-5561: -- Resolution: Fixed Fix Version/s: 2.2.1 0.23.10 2.3.0 3.0.0 Status: Resolved (was: Patch Available) Thanks for the contribution, Karthik, and thanks for the review Akira! I committed this to trunk, branch-2, branch-2.2, and branch-0.23. > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on > trunk > - > > Key: MAPREDUCE-5561 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Cindy Li >Assignee: Karthik Kambatla >Priority: Critical > Fix For: 3.0.0, 2.3.0, 0.23.10, 2.2.1 > > Attachments: mr-5561-1.patch, > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt > > > Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec > <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) > Time elapsed: 5.507 sec <<< FAILURE! > java.lang.AssertionError: expected: but was: > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:147) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802357#comment-13802357 ] Jason Lowe commented on MAPREDUCE-5561: --- +1, lgtm. It's a reasonable approach. The test failure is unrelated and most likely caused by something like YARN-1183. I tested the patch locally and it does not fail. I'll commit this shortly. > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on > trunk > - > > Key: MAPREDUCE-5561 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Cindy Li >Assignee: Karthik Kambatla >Priority: Critical > Attachments: mr-5561-1.patch, > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt > > > Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec > <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) > Time elapsed: 5.507 sec <<< FAILURE! > java.lang.AssertionError: expected: but was: > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:147) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802289#comment-13802289
]
Hadoop QA commented on MAPREDUCE-5561:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609720/mr-5561-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4130//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4130//console
This message is automatically generated.
> org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on
> trunk
> -
>
> Key: MAPREDUCE-5561
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Cindy Li
>Assignee: Karthik Kambatla
>Priority: Critical
> Attachments: mr-5561-1.patch,
> org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt
>
>
> Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl
> Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec
> <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl
> testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl)
> Time elapsed: 5.507 sec <<< FAILURE!
> java.lang.AssertionError: expected: but was:
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.failNotEquals(Assert.java:647)
> at org.junit.Assert.assertEquals(Assert.java:128)
> at org.junit.Assert.assertEquals(Assert.java:147)
> at
> org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817)
> at
> org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-5561: - Hadoop Flags: Reviewed > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on > trunk > - > > Key: MAPREDUCE-5561 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Cindy Li >Assignee: Karthik Kambatla >Priority: Critical > Attachments: mr-5561-1.patch, > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt > > > Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec > <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) > Time elapsed: 5.507 sec <<< FAILURE! > java.lang.AssertionError: expected: but was: > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:147) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802280#comment-13802280 ] Akira AJISAKA commented on MAPREDUCE-5561: -- +1, LGTM. I ran the test 3 times and haven't seen it fail. > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on > trunk > - > > Key: MAPREDUCE-5561 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Cindy Li >Assignee: Karthik Kambatla >Priority: Critical > Attachments: mr-5561-1.patch, > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt > > > Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec > <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) > Time elapsed: 5.507 sec <<< FAILURE! > java.lang.AssertionError: expected: but was: > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:147) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-5561: Target Version/s: 2.3.0 Status: Patch Available (was: Open) > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on > trunk > - > > Key: MAPREDUCE-5561 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Cindy Li >Assignee: Karthik Kambatla >Priority: Critical > Attachments: mr-5561-1.patch, > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt > > > Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec > <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) > Time elapsed: 5.507 sec <<< FAILURE! > java.lang.AssertionError: expected: but was: > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:147) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-5561: Attachment: mr-5561-1.patch Trivial fix that drops the assert. Ran the test multiple times and haven't seen it fail - I know that doesn't say much. [~jlowe], in this particular case, given the purpose of the test is to verify we eventually get to the FAILED state, I went ahead with the simpler approach. > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on > trunk > - > > Key: MAPREDUCE-5561 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Cindy Li >Assignee: Karthik Kambatla >Priority: Critical > Attachments: mr-5561-1.patch, > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt > > > Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec > <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) > Time elapsed: 5.507 sec <<< FAILURE! > java.lang.AssertionError: expected: but was: > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:147) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5397) AM crashes because Webapp failed to start on multi node cluster
[
https://issues.apache.org/jira/browse/MAPREDUCE-5397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802191#comment-13802191
]
Jian He commented on MAPREDUCE-5397:
[~jclum], I made a clean build and didn't reproduce this any more
> AM crashes because Webapp failed to start on multi node cluster
> ---
>
> Key: MAPREDUCE-5397
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5397
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Jian He
>Assignee: Jian He
> Attachments: log.txt
>
>
> I set up a 12 nodes cluster and tried submitting jobs but get this exception.
> But job is able to succeed after AM crashes and retry a few times(2 or 3)
> {code}
> 2013-07-12 18:56:28,438 INFO [main] org.mortbay.log: Extract
> jar:file:/grid/0/dev/jhe/hadoop-2.1.0-beta/share/hadoop/yarn/hadoop-yarn-common-2.1.0-beta.jar!/webapps/mapreduce
> to /tmp/Jetty_0_0_0_0_43554_mapreduceljbmlg/webapp
> 2013-07-12 18:56:28,528 WARN [main] org.mortbay.log: Failed startup of
> context
> org.mortbay.jetty.webapp.WebAppContext@2726b2{/,jar:file:/grid/0/dev/jhe/hadoop-2.1.0-beta/share/hadoop/yarn/hadoop-yarn-common-2.1.0-beta.jar!/webapps/mapreduce}
> java.io.FileNotFoundException:
> /tmp/Jetty_0_0_0_0_43554_mapreduceljbmlg/webapp/webapps/mapreduce/.keep
> (No such file or directory)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:194)
> at java.io.FileOutputStream.(FileOutputStream.java:145)
> at org.mortbay.resource.JarResource.extract(JarResource.java:215)
> at
> org.mortbay.jetty.webapp.WebAppContext.resolveWebApp(WebAppContext.java:974)
> at
> org.mortbay.jetty.webapp.WebAppContext.getWebInf(WebAppContext.java:832)
> at
> org.mortbay.jetty.webapp.WebInfConfiguration.configureClassLoader(WebInfConfiguration.java:62)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:489)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.hadoop.http.HttpServer.start(HttpServer.java:684)
> at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:211)
> at
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:134)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1019)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1394)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1390)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[
https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802190#comment-13802190
]
Hadoop QA commented on MAPREDUCE-4980:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609712/MAPREDUCE-4980--n7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 120
new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-tools/hadoop-gridmix hadoop-tools/hadoop-extras
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4129//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4129//console
This message is automatically generated.
> Parallel test execution of hadoop-mapreduce-client-core
> ---
>
> Key: MAPREDUCE-4980
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980
> Project: Hadoop Map/Reduce
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0
>Reporter: Tsuyoshi OZAWA
>Assignee: Andrey Klochkov
> Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch,
> MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch,
> MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch
>
>
> The maven surefire plugin supports parallel testing feature. By using it, the
> tests can be run more faster.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated MAPREDUCE-4980: --- Attachment: MAPREDUCE-4980--n7.patch Rebased the patch. > Parallel test execution of hadoop-mapreduce-client-core > --- > > Key: MAPREDUCE-4980 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 3.0.0 >Reporter: Tsuyoshi OZAWA >Assignee: Andrey Klochkov > Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch, > MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch, > MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch > > > The maven surefire plugin supports parallel testing feature. By using it, the > tests can be run more faster. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated MAPREDUCE-4980: --- Attachment: (was: MAPREDUCE-4980--n7.patch) > Parallel test execution of hadoop-mapreduce-client-core > --- > > Key: MAPREDUCE-4980 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 3.0.0 >Reporter: Tsuyoshi OZAWA >Assignee: Andrey Klochkov > Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch, > MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch, > MAPREDUCE-4980.patch > > > The maven surefire plugin supports parallel testing feature. By using it, the > tests can be run more faster. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5221) Reduce side Combiner is not used when using the new API
[ https://issues.apache.org/jira/browse/MAPREDUCE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802016#comment-13802016 ] Tsuyoshi OZAWA commented on MAPREDUCE-5221: --- I think almost work is done. I'll rebase the latest patch soon and wait for review. > Reduce side Combiner is not used when using the new API > --- > > Key: MAPREDUCE-5221 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5221 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.0.4-alpha >Reporter: Siddharth Seth >Assignee: Tsuyoshi OZAWA > Attachments: MAPREDUCE-5221.1.patch, MAPREDUCE-5221.2.patch, > MAPREDUCE-5221.3.patch, MAPREDUCE-5221.4.patch, MAPREDUCE-5221.5.patch, > MAPREDUCE-5221.6.patch > > > If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this > will silently ignored on the reduce side since the reduce side usage is only > aware of the old api combiner. > This doesn't fail the job - since the new combiner key does not deprecate the > old key. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13801931#comment-13801931 ] Ravi Prakash commented on MAPREDUCE-3860: - Hi Andrey! Thanks for your answers. 1. It will wait for 10s (100ms * 100), but you're right, that is tolerable too. 2. Aah! I did not know that. Thanks. 3. I agree that username should not be null after anonymization. So maybe we should remove the line "anonymizedJob.getUser() != null &&" 4. I can see it either way and I'm not a stickler about either way. I'm fine with how it is. I would probaly classify all my comments as nits above. So I'm fine with this patch. I'll let a committer make the final judgement call. Thanks Andrey! > [Rumen] Bring back the removed Rumen unit tests > --- > > Key: MAPREDUCE-3860 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tools/rumen >Reporter: Ravi Gummadi >Assignee: Andrey Klochkov > Attachments: MAPREDUCE-3860--n2.patch, MAPREDUCE-3860.patch, > rumen-test-data.tar.gz > > > MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder > and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need > to be brought back: > TestZombieJob.java > TestRumenJobTraces.java > TestRumenFolder.java > TestRumenAnonymization.java > TestParsedLine.java > TestConcurrentRead.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5221) Reduce side Combiner is not used when using the new API
[ https://issues.apache.org/jira/browse/MAPREDUCE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13801907#comment-13801907 ] Jonathan Eagles commented on MAPREDUCE-5221: I haven't seen any update on this ticket in some time. Can a comment on the current status be provided we can get this improvement in? > Reduce side Combiner is not used when using the new API > --- > > Key: MAPREDUCE-5221 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5221 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.0.4-alpha >Reporter: Siddharth Seth >Assignee: Tsuyoshi OZAWA > Attachments: MAPREDUCE-5221.1.patch, MAPREDUCE-5221.2.patch, > MAPREDUCE-5221.3.patch, MAPREDUCE-5221.4.patch, MAPREDUCE-5221.5.patch, > MAPREDUCE-5221.6.patch > > > If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this > will silently ignored on the reduce side since the reduce side usage is only > aware of the old api combiner. > This doesn't fail the job - since the new combiner key does not deprecate the > old key. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13801874#comment-13801874 ] Jason Lowe commented on MAPREDUCE-5561: --- Yes, the test is definitely racy. There's no guarantee the job will be in the FAIL_ABORT state while when we look at it asynchronously. A couple of approaches to fixing this: # As [~kkambatl] points out, we can skip the FAIL_ABORT check. The real purpose of this test is to verify we eventually get to the FAILED state without hanging when tasks fail. # A more deterministic, explicit test for FAIL_ABORT would be to use an output committer with a barrier, similar to TestingOutputCommitter but with the barrier in the abortJob method, so we can guarantee the job will pause in the FAIL_ABORT state. Then we can release the committer from the barrier and verify the job proceeds to failed. > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on > trunk > - > > Key: MAPREDUCE-5561 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Cindy Li >Assignee: Karthik Kambatla >Priority: Critical > Attachments: > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt > > > Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec > <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl > testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) > Time elapsed: 5.507 sec <<< FAILURE! > java.lang.AssertionError: expected: but was: > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:147) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5397) AM crashes because Webapp failed to start on multi node cluster
[
https://issues.apache.org/jira/browse/MAPREDUCE-5397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13801816#comment-13801816
]
Josh Clum commented on MAPREDUCE-5397:
--
[~jianhe] - I'm getting the same error. Have you made any progress debugging?
> AM crashes because Webapp failed to start on multi node cluster
> ---
>
> Key: MAPREDUCE-5397
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5397
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Jian He
>Assignee: Jian He
> Attachments: log.txt
>
>
> I set up a 12 nodes cluster and tried submitting jobs but get this exception.
> But job is able to succeed after AM crashes and retry a few times(2 or 3)
> {code}
> 2013-07-12 18:56:28,438 INFO [main] org.mortbay.log: Extract
> jar:file:/grid/0/dev/jhe/hadoop-2.1.0-beta/share/hadoop/yarn/hadoop-yarn-common-2.1.0-beta.jar!/webapps/mapreduce
> to /tmp/Jetty_0_0_0_0_43554_mapreduceljbmlg/webapp
> 2013-07-12 18:56:28,528 WARN [main] org.mortbay.log: Failed startup of
> context
> org.mortbay.jetty.webapp.WebAppContext@2726b2{/,jar:file:/grid/0/dev/jhe/hadoop-2.1.0-beta/share/hadoop/yarn/hadoop-yarn-common-2.1.0-beta.jar!/webapps/mapreduce}
> java.io.FileNotFoundException:
> /tmp/Jetty_0_0_0_0_43554_mapreduceljbmlg/webapp/webapps/mapreduce/.keep
> (No such file or directory)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:194)
> at java.io.FileOutputStream.(FileOutputStream.java:145)
> at org.mortbay.resource.JarResource.extract(JarResource.java:215)
> at
> org.mortbay.jetty.webapp.WebAppContext.resolveWebApp(WebAppContext.java:974)
> at
> org.mortbay.jetty.webapp.WebAppContext.getWebInf(WebAppContext.java:832)
> at
> org.mortbay.jetty.webapp.WebInfConfiguration.configureClassLoader(WebInfConfiguration.java:62)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:489)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at org.apache.hadoop.http.HttpServer.start(HttpServer.java:684)
> at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:211)
> at
> org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:134)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1019)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1394)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1390)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
