[jira] [Commented] (MAPREDUCE-5485) Allow repeating job commit by extending OutputCommitter API
[ https://issues.apache.org/jira/browse/MAPREDUCE-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14956130#comment-14956130 ] Nemon Lou commented on MAPREDUCE-5485: -- Feel free to take it. And thanks for your interest. > Allow repeating job commit by extending OutputCommitter API > --- > > Key: MAPREDUCE-5485 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5485 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Affects Versions: 2.1.0-beta >Reporter: Nemon Lou >Assignee: Junping Du > > There are chances MRAppMaster crush during job committing,or NodeManager > restart cause the committing AM exit due to container expire.In these cases > ,the job will fail. > However,some jobs can redo commit so failing the job becomes unnecessary. > Let clients tell AM to allow redo commit or not is a better choice. > This idea comes from Jason Lowe's comments in MAPREDUCE-4819 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5485) Allow repeating job commit by extending OutputCommitter API
[ https://issues.apache.org/jira/browse/MAPREDUCE-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated MAPREDUCE-5485: - Assignee: (was: Nemon Lou) Allow repeating job commit by extending OutputCommitter API --- Key: MAPREDUCE-5485 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5485 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.1.0-beta Reporter: Nemon Lou There are chances MRAppMaster crush during job committing,or NodeManager restart cause the committing AM exit due to container expire.In these cases ,the job will fail. However,some jobs can redo commit so failing the job becomes unnecessary. Let clients tell AM to allow redo commit or not is a better choice. This idea comes from Jason Lowe's comments in MAPREDUCE-4819 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (MAPREDUCE-5581) killing jobs which have failed causes log missing
Nemon Lou created MAPREDUCE-5581: Summary: killing jobs which have failed causes log missing Key: MAPREDUCE-5581 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5581 Project: Hadoop Map/Reduce Issue Type: Bug Components: client Affects Versions: 2.1.1-beta Reporter: Nemon Lou In hive code,when a job failed,they invoke the RunningJob.killJob() API . From mapreduce client side,when job is at failed state,the YARNRunner will invoke resMgrDelegate.killApplication to kill that job.And this causes logs missing from job history server. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5581) killing jobs which have failed causes log missing
[ https://issues.apache.org/jira/browse/MAPREDUCE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated MAPREDUCE-5581: - Description: In hive code,when a job failed,they invoke the RunningJob.killJob() API immediately. From mapreduce client side,when job is at failed state,the YARNRunner will invoke resMgrDelegate.killApplication to kill that job.And this prevent AM from writing logs to job history server. was: In hive code,when a job failed,they invoke the RunningJob.killJob() API . From mapreduce client side,when job is at failed state,the YARNRunner will invoke resMgrDelegate.killApplication to kill that job.And this causes logs missing from job history server. killing jobs which have failed causes log missing - Key: MAPREDUCE-5581 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5581 Project: Hadoop Map/Reduce Issue Type: Bug Components: client Affects Versions: 2.1.1-beta Reporter: Nemon Lou In hive code,when a job failed,they invoke the RunningJob.killJob() API immediately. From mapreduce client side,when job is at failed state,the YARNRunner will invoke resMgrDelegate.killApplication to kill that job.And this prevent AM from writing logs to job history server. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5485) Allow repeating job commit by extending OutputCommitter API
[ https://issues.apache.org/jira/browse/MAPREDUCE-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated MAPREDUCE-5485: - Summary: Allow repeating job commit by extending OutputCommitter API (was: Allow repeating job commit by extending OutputCommiter API) Allow repeating job commit by extending OutputCommitter API --- Key: MAPREDUCE-5485 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5485 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.1.0-beta Reporter: Nemon Lou There are chances MRAppMaster crush during job committing,or NodeManager restart cause the committing AM exit due to container expire.In these cases ,the job will fail. However,some jobs can redo commit so failing the job becomes unnecessary. Let clients tell AM to allow redo commit or not is a better choice. This idea comes from Jason Lowe's comments in MAPREDUCE-4819 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5485) Allow repeating job commit by extending OutputCommitter API
[ https://issues.apache.org/jira/browse/MAPREDUCE-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated MAPREDUCE-5485: - Assignee: Nemon Lou Allow repeating job commit by extending OutputCommitter API --- Key: MAPREDUCE-5485 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5485 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.1.0-beta Reporter: Nemon Lou Assignee: Nemon Lou There are chances MRAppMaster crush during job committing,or NodeManager restart cause the committing AM exit due to container expire.In these cases ,the job will fail. However,some jobs can redo commit so failing the job becomes unnecessary. Let clients tell AM to allow redo commit or not is a better choice. This idea comes from Jason Lowe's comments in MAPREDUCE-4819 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5485) Allow repeating job commit by extending OutputCommitter API
[ https://issues.apache.org/jira/browse/MAPREDUCE-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764207#comment-13764207 ] Nemon Lou commented on MAPREDUCE-5485: -- Some initial thoughts: 1,Adding a method : boolean isCommitJobRepeatable() for outputCommitters. The abstract class OutputCommitter will return false for it. And FileOutputCommitter returns true,as FileOutputCommitter's commitJob will remove exists files that will be committed and then do the rename. For commit repeatable jobs,we have 2,3,4: 2,When commitJob method throws exception,AM will retry commit directly with a times limit. 3,When AM has an error during committing(error is not from the commitJob method),it will not reach a job final state ,but just exit and leave work to another AM. 4,For secondly started AM,it will check the phase the job has reached.If the phase is commit failed,then it's state will reach job committing after recovery and start commit again. Allow repeating job commit by extending OutputCommitter API --- Key: MAPREDUCE-5485 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5485 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.1.0-beta Reporter: Nemon Lou Assignee: Nemon Lou There are chances MRAppMaster crush during job committing,or NodeManager restart cause the committing AM exit due to container expire.In these cases ,the job will fail. However,some jobs can redo commit so failing the job becomes unnecessary. Let clients tell AM to allow redo commit or not is a better choice. This idea comes from Jason Lowe's comments in MAPREDUCE-4819 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5164) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS
[ https://issues.apache.org/jira/browse/MAPREDUCE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated MAPREDUCE-5164: - Attachment: MAPREDUCE-5164.patch Patch updated, HADOOP_CLIENT_OPTS is added for windows CMD, too. command mapred job and mapred queue omit HADOOP_CLIENT_OPTS - Key: MAPREDUCE-5164 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5164 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Nemon Lou Assignee: Nemon Lou Attachments: MAPREDUCE-5164.patch, MAPREDUCE-5164.patch, MAPREDUCE-5164.patch, MAPREDUCE-5164.patch HADOOP_CLIENT_OPTS does not take effect when type mapred job -list and mapred queue -list. The mapred script omit it -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5164) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS
[ https://issues.apache.org/jira/browse/MAPREDUCE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated MAPREDUCE-5164: - Status: Patch Available (was: Open) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS - Key: MAPREDUCE-5164 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5164 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Nemon Lou Assignee: Nemon Lou Attachments: MAPREDUCE-5164.patch, MAPREDUCE-5164.patch, MAPREDUCE-5164.patch, MAPREDUCE-5164.patch HADOOP_CLIENT_OPTS does not take effect when type mapred job -list and mapred queue -list. The mapred script omit it -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5485) Allow repeating job commit by extending OutputCommiter API
Nemon Lou created MAPREDUCE-5485: Summary: Allow repeating job commit by extending OutputCommiter API Key: MAPREDUCE-5485 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5485 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.1.0-beta Reporter: Nemon Lou There are chances MRAppMaster crush during job committing,or NodeManager restart cause the committing AM exit due to container expire.In these cases ,the job will fail. However,some jobs can redo commit so failing the job becomes unnecessary. Let clients tell AM to allow redo commit or not is a better choice. This idea comes from Jason Lowe's comments in MAPREDUCE-4819 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5164) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS
[ https://issues.apache.org/jira/browse/MAPREDUCE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated MAPREDUCE-5164: - Status: Open (was: Patch Available) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS - Key: MAPREDUCE-5164 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5164 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Nemon Lou Assignee: Nemon Lou Attachments: MAPREDUCE-5164.patch HADOOP_CLIENT_OPTS does not take effect when type mapred job -list and mapred queue -list. The mapred script omit it -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5164) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS
[ https://issues.apache.org/jira/browse/MAPREDUCE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated MAPREDUCE-5164: - Attachment: MAPREDUCE-5164.patch Change into the same way as HADOOP-8623 does. command mapred job and mapred queue omit HADOOP_CLIENT_OPTS - Key: MAPREDUCE-5164 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5164 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Nemon Lou Assignee: Nemon Lou Attachments: MAPREDUCE-5164.patch, MAPREDUCE-5164.patch HADOOP_CLIENT_OPTS does not take effect when type mapred job -list and mapred queue -list. The mapred script omit it -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5164) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS
[ https://issues.apache.org/jira/browse/MAPREDUCE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated MAPREDUCE-5164: - Status: Patch Available (was: Open) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS - Key: MAPREDUCE-5164 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5164 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Nemon Lou Assignee: Nemon Lou Attachments: MAPREDUCE-5164.patch, MAPREDUCE-5164.patch, MAPREDUCE-5164.patch HADOOP_CLIENT_OPTS does not take effect when type mapred job -list and mapred queue -list. The mapred script omit it -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5164) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS
[ https://issues.apache.org/jira/browse/MAPREDUCE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated MAPREDUCE-5164: - Attachment: MAPREDUCE-5164.patch Notice that JobHistoryServer is also in this script,this patch is invalid.Changing back again.Sorry for that. command mapred job and mapred queue omit HADOOP_CLIENT_OPTS - Key: MAPREDUCE-5164 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5164 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Nemon Lou Assignee: Nemon Lou Attachments: MAPREDUCE-5164.patch, MAPREDUCE-5164.patch, MAPREDUCE-5164.patch HADOOP_CLIENT_OPTS does not take effect when type mapred job -list and mapred queue -list. The mapred script omit it -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5469) Counters for MRAppMaster
nemon lou created MAPREDUCE-5469: Summary: Counters for MRAppMaster Key: MAPREDUCE-5469 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5469 Project: Hadoop Map/Reduce Issue Type: New Feature Affects Versions: 2.0.5-alpha Reporter: nemon lou We have counters for map tasks and reduce tasks ,but has no counters for MRAppMaster. Sometimes we need information like GC time,memory usage for AM tuning. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5469) Counters for MRAppMaster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nemon lou updated MAPREDUCE-5469: - Issue Type: Improvement (was: New Feature) Counters for MRAppMaster Key: MAPREDUCE-5469 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5469 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.0.5-alpha Reporter: nemon lou We have counters for map tasks and reduce tasks ,but has no counters for MRAppMaster. Sometimes we need information like GC time,memory usage for AM tuning. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4413) MR lib dir contains jdiff (which is gpl)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nemon lou updated MAPREDUCE-4413: - Attachment: MAPREDUCE-4413.patch MR lib dir contains jdiff (which is gpl) Key: MAPREDUCE-4413 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4413 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: nemon lou Priority: Critical Attachments: MAPREDUCE-4413.patch, MAPREDUCE-4413.patch A tarball built from trunk contains the following: ./share/hadoop/mapreduce/lib/jdiff-1.0.9.jar jdiff is gplv2, we need to exclude it from the build artifact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4413) MR lib dir contains jdiff (which is gpl)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nemon lou updated MAPREDUCE-4413: - Status: Open (was: Patch Available) MR lib dir contains jdiff (which is gpl) Key: MAPREDUCE-4413 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4413 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: nemon lou Priority: Critical Attachments: MAPREDUCE-4413.patch, MAPREDUCE-4413.patch A tarball built from trunk contains the following: ./share/hadoop/mapreduce/lib/jdiff-1.0.9.jar jdiff is gplv2, we need to exclude it from the build artifact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4413) MR lib dir contains jdiff (which is gpl)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nemon lou updated MAPREDUCE-4413: - Status: Patch Available (was: Open) MR lib dir contains jdiff (which is gpl) Key: MAPREDUCE-4413 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4413 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: nemon lou Priority: Critical Attachments: MAPREDUCE-4413.patch, MAPREDUCE-4413.patch A tarball built from trunk contains the following: ./share/hadoop/mapreduce/lib/jdiff-1.0.9.jar jdiff is gplv2, we need to exclude it from the build artifact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4413) MR lib dir contains jdiff (which is gpl)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nemon lou updated MAPREDUCE-4413: - Target Version/s: (was: ) Status: Patch Available (was: Open) MR lib dir contains jdiff (which is gpl) Key: MAPREDUCE-4413 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4413 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: nemon lou Priority: Critical Attachments: MAPREDUCE-4413.patch A tarball built from trunk contains the following: ./share/hadoop/mapreduce/lib/jdiff-1.0.9.jar jdiff is gplv2, we need to exclude it from the build artifact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4413) MR lib dir contains jdiff (which is gpl)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nemon lou updated MAPREDUCE-4413: - Assignee: nemon lou MR lib dir contains jdiff (which is gpl) Key: MAPREDUCE-4413 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4413 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: nemon lou Priority: Critical A tarball built from trunk contains the following: ./share/hadoop/mapreduce/lib/jdiff-1.0.9.jar jdiff is gplv2, we need to exclude it from the build artifact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4413) MR lib dir contains jdiff (which is gpl)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nemon lou updated MAPREDUCE-4413: - Attachment: MAPREDUCE-4413.patch Change hadoop-assemblies settings to exclude jdiff for MapReduce distribution. MR lib dir contains jdiff (which is gpl) Key: MAPREDUCE-4413 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4413 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: nemon lou Priority: Critical Attachments: MAPREDUCE-4413.patch A tarball built from trunk contains the following: ./share/hadoop/mapreduce/lib/jdiff-1.0.9.jar jdiff is gplv2, we need to exclude it from the build artifact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5427) TestRMContainerAllocator.testUpdatedNodes fails on jdk7
nemon lou created MAPREDUCE-5427:
Summary: TestRMContainerAllocator.testUpdatedNodes fails on jdk7
Key: MAPREDUCE-5427
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5427
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.1.0-beta, 2.0.5-alpha
Reporter: nemon lou
{code}
---
Test set: org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator
---
Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 42.777 sec
FAILURE!
testUpdatedNodes(org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator)
Time elapsed: 0.14 sec FAILURE!
junit.framework.AssertionFailedError: null
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at
org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator.testUpdatedNodes(TestRMContainerAllocator.java:747)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5427) TestRMContainerAllocator.testUpdatedNodes fails on jdk7
[
https://issues.apache.org/jira/browse/MAPREDUCE-5427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13722242#comment-13722242
]
nemon lou commented on MAPREDUCE-5427:
--
When test case testUpdatedNodes runs before testUpdatedNodes ,then the test
will fail.
They share the same properties in MyContainerAllocator,which are static.
TestRMContainerAllocator.testUpdatedNodes fails on jdk7
---
Key: MAPREDUCE-5427
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5427
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.1.0-beta, 2.0.5-alpha
Reporter: nemon lou
Labels: java7
{code}
---
Test set: org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator
---
Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 42.777 sec
FAILURE!
testUpdatedNodes(org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator)
Time elapsed: 0.14 sec FAILURE!
junit.framework.AssertionFailedError: null
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at
org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator.testUpdatedNodes(TestRMContainerAllocator.java:747)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4579) TestTaskAttempt fails jdk7
[ https://issues.apache.org/jira/browse/MAPREDUCE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718054#comment-13718054 ] nemon lou commented on MAPREDUCE-4579: -- Test case org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt still can fail after apply this patch. TestTaskAttempt fails jdk7 -- Key: MAPREDUCE-4579 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4579 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.3, 3.0.0, 2.0.2-alpha Reporter: Thomas Graves Assignee: Thomas Graves Labels: java7 Fix For: 3.0.0, 2.0.2-alpha Attachments: MAPREDUCE-4579.patch --- Test set: org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt --- Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.205 sec FAILURE!testAttemptContainerRequest(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt) Time elapsed: 0.032 sec ERROR! java.io.EOFException at java.io.DataInputStream.readByte(DataInputStream.java:267) at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:308) at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:329) at org.apache.hadoop.io.Text.readFields(Text.java:280) at org.apache.hadoop.security.token.Token.readFields(Token.java:165) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4579) TestTaskAttempt fails jdk7
[
https://issues.apache.org/jira/browse/MAPREDUCE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718064#comment-13718064
]
nemon lou commented on MAPREDUCE-4579:
--
Test case org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt still can
fail after apply this patch.
It fails once in a while when you run all of them together.
But if you run a single test case,taking testDoubleTooManyFetchFailure for
example,it will fail without doubt.
I will fire a jira if it's an issue .
{code:xml}
Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 7.893 sec
FAILURE!
Results :
Tests in error:
testLaunchFailedWhileKilling(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileRunning(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileCommitting(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testDoubleTooManyFetchFailure(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0
{code}
TestTaskAttempt fails jdk7
--
Key: MAPREDUCE-4579
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4579
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.23.3, 3.0.0, 2.0.2-alpha
Reporter: Thomas Graves
Assignee: Thomas Graves
Labels: java7
Fix For: 3.0.0, 2.0.2-alpha
Attachments: MAPREDUCE-4579.patch
---
Test set: org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
---
Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.205 sec
FAILURE!testAttemptContainerRequest(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
Time elapsed: 0.032 sec ERROR!
java.io.EOFException
at java.io.DataInputStream.readByte(DataInputStream.java:267)
at
org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:308)
at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:329)
at org.apache.hadoop.io.Text.readFields(Text.java:280)
at org.apache.hadoop.security.token.Token.readFields(Token.java:165)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4579) TestTaskAttempt fails jdk7
[
https://issues.apache.org/jira/browse/MAPREDUCE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718066#comment-13718066
]
nemon lou commented on MAPREDUCE-4579:
--
logs for running
org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt#testContainerCleanedWhileRunning
{code:xml}
testcase time=0.057
classname=org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
name=testContainerCleanedWhileRunning
error type=java.lang.NullPointerExceptionjava.lang.NullPointerException
at org.apache.hadoop.security.token.Token.write(Token.java:216)
at
org.apache.hadoop.mapred.ShuffleHandler.serializeServiceData(ShuffleHandler.java:205)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createCommonContainerLaunchContext(TaskAttemptImpl.java:695)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createContainerLaunchContext(TaskAttemptImpl.java:751)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1309)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1282)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1009)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt.testContainerCleanedWhileRunning(TestTaskAttempt.java:410)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
/error
system-out2013-07-24 10:32:27,664 INFO [main] util.RackResolver
(RackResolver.java:coreResolve(100)) - Resolved 127.0.0.1 to /default-rack
2013-07-24 10:32:27,665 INFO [main] impl.TaskAttemptImpl
(TaskAttemptImpl.java:handle(1020)) - attempt_1_0002_m_01_1 TaskAttempt
Transitioned from NEW to UNASSIGNED
2013-07-24 10:32:27,666 INFO [main] util.RackResolver
(RackResolver.java:coreResolve(100)) - Resolved 127.0.0.1 to /default-rack
2013-07-24 10:32:27,668
[jira] [Created] (MAPREDUCE-5414) TestTaskAttempt fails jdk7 with NullPointerException
nemon lou created MAPREDUCE-5414:
Summary: TestTaskAttempt fails jdk7 with NullPointerException
Key: MAPREDUCE-5414
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5414
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.0.5-alpha
Reporter: nemon lou
Assignee: nemon lou
Test case org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt fails
once in a while when i run all of them together.
{code:xml}
Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 7.893 sec
FAILURE!
Results :
Tests in error:
testLaunchFailedWhileKilling(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileRunning(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileCommitting(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testDoubleTooManyFetchFailure(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0
{code}
But if i run a single test case,taking testContainerCleanedWhileRunning for
example,it will fail without doubt.
{code:xml}
testcase time=0.057
classname=org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
name=testContainerCleanedWhileRunning
error type=java.lang.NullPointerExceptionjava.lang.NullPointerException
at org.apache.hadoop.security.token.Token.write(Token.java:216)
at
org.apache.hadoop.mapred.ShuffleHandler.serializeServiceData(ShuffleHandler.java:205)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createCommonContainerLaunchContext(TaskAttemptImpl.java:695)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createContainerLaunchContext(TaskAttemptImpl.java:751)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1309)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1282)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1009)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt.testContainerCleanedWhileRunning(TestTaskAttempt.java:410)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
[jira] [Commented] (MAPREDUCE-4579) TestTaskAttempt fails jdk7
[ https://issues.apache.org/jira/browse/MAPREDUCE-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718101#comment-13718101 ] nemon lou commented on MAPREDUCE-4579: -- I have created MAPREDUCE-5414. TestTaskAttempt fails jdk7 -- Key: MAPREDUCE-4579 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4579 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.3, 3.0.0, 2.0.2-alpha Reporter: Thomas Graves Assignee: Thomas Graves Labels: java7 Fix For: 3.0.0, 2.0.2-alpha Attachments: MAPREDUCE-4579.patch --- Test set: org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt --- Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.205 sec FAILURE!testAttemptContainerRequest(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt) Time elapsed: 0.032 sec ERROR! java.io.EOFException at java.io.DataInputStream.readByte(DataInputStream.java:267) at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:308) at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:329) at org.apache.hadoop.io.Text.readFields(Text.java:280) at org.apache.hadoop.security.token.Token.readFields(Token.java:165) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5414) TestTaskAttempt fails jdk7 with NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nemon lou updated MAPREDUCE-5414:
-
Attachment: MAPREDUCE-5414.patch
TestTaskAttempt fails jdk7 with NullPointerException
Key: MAPREDUCE-5414
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5414
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.0.5-alpha
Reporter: nemon lou
Assignee: nemon lou
Labels: java7
Attachments: MAPREDUCE-5414.patch
Test case org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt fails
once in a while when i run all of them together.
{code:xml}
Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 7.893 sec
FAILURE!
Results :
Tests in error:
testLaunchFailedWhileKilling(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileRunning(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileCommitting(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testDoubleTooManyFetchFailure(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0
{code}
But if i run a single test case,taking testContainerCleanedWhileRunning for
example,it will fail without doubt.
{code:xml}
testcase time=0.057
classname=org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
name=testContainerCleanedWhileRunning
error
type=java.lang.NullPointerExceptionjava.lang.NullPointerException
at org.apache.hadoop.security.token.Token.write(Token.java:216)
at
org.apache.hadoop.mapred.ShuffleHandler.serializeServiceData(ShuffleHandler.java:205)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createCommonContainerLaunchContext(TaskAttemptImpl.java:695)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createContainerLaunchContext(TaskAttemptImpl.java:751)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1309)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1282)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1009)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt.testContainerCleanedWhileRunning(TestTaskAttempt.java:410)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at
[jira] [Updated] (MAPREDUCE-5414) TestTaskAttempt fails jdk7 with NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nemon lou updated MAPREDUCE-5414:
-
Status: Patch Available (was: Open)
TestTaskAttempt fails jdk7 with NullPointerException
Key: MAPREDUCE-5414
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5414
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.0.5-alpha
Reporter: nemon lou
Assignee: nemon lou
Labels: java7
Attachments: MAPREDUCE-5414.patch
Test case org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt fails
once in a while when i run all of them together.
{code:xml}
Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 7.893 sec
FAILURE!
Results :
Tests in error:
testLaunchFailedWhileKilling(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileRunning(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileCommitting(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testDoubleTooManyFetchFailure(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0
{code}
But if i run a single test case,taking testContainerCleanedWhileRunning for
example,it will fail without doubt.
{code:xml}
testcase time=0.057
classname=org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
name=testContainerCleanedWhileRunning
error
type=java.lang.NullPointerExceptionjava.lang.NullPointerException
at org.apache.hadoop.security.token.Token.write(Token.java:216)
at
org.apache.hadoop.mapred.ShuffleHandler.serializeServiceData(ShuffleHandler.java:205)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createCommonContainerLaunchContext(TaskAttemptImpl.java:695)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createContainerLaunchContext(TaskAttemptImpl.java:751)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1309)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1282)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1009)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt.testContainerCleanedWhileRunning(TestTaskAttempt.java:410)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at
[jira] [Updated] (MAPREDUCE-5414) TestTaskAttempt fails jdk7 with NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nemon lou updated MAPREDUCE-5414:
-
Status: Open (was: Patch Available)
TestTaskAttempt fails jdk7 with NullPointerException
Key: MAPREDUCE-5414
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5414
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.0.5-alpha
Reporter: nemon lou
Assignee: nemon lou
Labels: java7
Attachments: MAPREDUCE-5414.patch
Test case org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt fails
once in a while when i run all of them together.
{code:xml}
Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 7.893 sec
FAILURE!
Results :
Tests in error:
testLaunchFailedWhileKilling(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileRunning(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileCommitting(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testDoubleTooManyFetchFailure(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0
{code}
But if i run a single test case,taking testContainerCleanedWhileRunning for
example,it will fail without doubt.
{code:xml}
testcase time=0.057
classname=org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
name=testContainerCleanedWhileRunning
error
type=java.lang.NullPointerExceptionjava.lang.NullPointerException
at org.apache.hadoop.security.token.Token.write(Token.java:216)
at
org.apache.hadoop.mapred.ShuffleHandler.serializeServiceData(ShuffleHandler.java:205)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createCommonContainerLaunchContext(TaskAttemptImpl.java:695)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createContainerLaunchContext(TaskAttemptImpl.java:751)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1309)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1282)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1009)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt.testContainerCleanedWhileRunning(TestTaskAttempt.java:410)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at
[jira] [Updated] (MAPREDUCE-5414) TestTaskAttempt fails jdk7 with NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nemon lou updated MAPREDUCE-5414:
-
Status: Patch Available (was: Open)
TestTaskAttempt fails jdk7 with NullPointerException
Key: MAPREDUCE-5414
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5414
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.0.5-alpha
Reporter: nemon lou
Assignee: nemon lou
Labels: java7
Attachments: MAPREDUCE-5414.patch, MAPREDUCE-5414.patch
Test case org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt fails
once in a while when i run all of them together.
{code:xml}
Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 7.893 sec
FAILURE!
Results :
Tests in error:
testLaunchFailedWhileKilling(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileRunning(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileCommitting(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testDoubleTooManyFetchFailure(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0
{code}
But if i run a single test case,taking testContainerCleanedWhileRunning for
example,it will fail without doubt.
{code:xml}
testcase time=0.057
classname=org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
name=testContainerCleanedWhileRunning
error
type=java.lang.NullPointerExceptionjava.lang.NullPointerException
at org.apache.hadoop.security.token.Token.write(Token.java:216)
at
org.apache.hadoop.mapred.ShuffleHandler.serializeServiceData(ShuffleHandler.java:205)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createCommonContainerLaunchContext(TaskAttemptImpl.java:695)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createContainerLaunchContext(TaskAttemptImpl.java:751)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1309)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1282)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1009)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt.testContainerCleanedWhileRunning(TestTaskAttempt.java:410)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at
[jira] [Updated] (MAPREDUCE-5414) TestTaskAttempt fails jdk7 with NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nemon lou updated MAPREDUCE-5414:
-
Attachment: MAPREDUCE-5414.patch
Updating patch as Ravi Prakash suggested.Thanks for your advise.
TestTaskAttempt fails jdk7 with NullPointerException
Key: MAPREDUCE-5414
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5414
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.0.5-alpha
Reporter: nemon lou
Assignee: nemon lou
Labels: java7
Attachments: MAPREDUCE-5414.patch, MAPREDUCE-5414.patch
Test case org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt fails
once in a while when i run all of them together.
{code:xml}
Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 7.893 sec
FAILURE!
Results :
Tests in error:
testLaunchFailedWhileKilling(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileRunning(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testContainerCleanedWhileCommitting(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
testDoubleTooManyFetchFailure(org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt)
Tests run: 9, Failures: 0, Errors: 4, Skipped: 0
{code}
But if i run a single test case,taking testContainerCleanedWhileRunning for
example,it will fail without doubt.
{code:xml}
testcase time=0.057
classname=org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
name=testContainerCleanedWhileRunning
error
type=java.lang.NullPointerExceptionjava.lang.NullPointerException
at org.apache.hadoop.security.token.Token.write(Token.java:216)
at
org.apache.hadoop.mapred.ShuffleHandler.serializeServiceData(ShuffleHandler.java:205)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createCommonContainerLaunchContext(TaskAttemptImpl.java:695)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createContainerLaunchContext(TaskAttemptImpl.java:751)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1309)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1282)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:357)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:298)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1009)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt.testContainerCleanedWhileRunning(TestTaskAttempt.java:410)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at
[jira] [Commented] (MAPREDUCE-4071) NPE while executing MRAppMaster shutdown hook
[
https://issues.apache.org/jira/browse/MAPREDUCE-4071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13673022#comment-13673022
]
nemon lou commented on MAPREDUCE-4071:
--
When split metadata size exceeds 1000,MRAppMaster will throw an IOException
during start and the NPE shows up.
Bhallamudi,are you still working on this?
NPE while executing MRAppMaster shutdown hook
-
Key: MAPREDUCE-4071
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4071
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am, mrv2
Affects Versions: 0.23.3, 2.0.0-alpha, trunk
Reporter: Bhallamudi Venkata Siva Kamesh
Attachments: MAPREDUCE-4071-1.patch, MAPREDUCE-4071-2.patch,
MAPREDUCE-4071-2.patch, MAPREDUCE-4071.patch
While running the shutdown hook of MRAppMaster, hit NPE
{noformat}
Exception in thread Thread-1 java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.setSignalled(MRAppMaster.java:668)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1004)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-259) Rack-aware Shuffle
[
https://issues.apache.org/jira/browse/MAPREDUCE-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13666926#comment-13666926
]
nemon lou commented on MAPREDUCE-259:
-
Interesting featrue. Still working on this?
Rack-aware Shuffle
--
Key: MAPREDUCE-259
URL: https://issues.apache.org/jira/browse/MAPREDUCE-259
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Arun C Murthy
Assignee: Arun C Murthy
We could try and experiment with *rack-aware* scheduling of fetches
per-reducer. Given the disparities between in-rack and off-rack bandwidth it
could be a improvement to do something along these lines:
{noformat}
if (no. of known map-output locations than no. of copier threads) {
try to schedule 75% of copies off-rack
try schedule 25% of copies in-rack
}
{noformat}
This could lead to better utilization of both in-rack switch b/w...
Clearly we want to schedule more cross-switch than in-rack since off-rack
copies will take significantly more time; hence the 75-25 split.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-259) Rack-aware Shuffle
[
https://issues.apache.org/jira/browse/MAPREDUCE-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13666927#comment-13666927
]
nemon lou commented on MAPREDUCE-259:
-
MAPREDUCE-2038 is doing the same thing,but is not updated for a long time,too.
Rack-aware Shuffle
--
Key: MAPREDUCE-259
URL: https://issues.apache.org/jira/browse/MAPREDUCE-259
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Arun C Murthy
Assignee: Arun C Murthy
We could try and experiment with *rack-aware* scheduling of fetches
per-reducer. Given the disparities between in-rack and off-rack bandwidth it
could be a improvement to do something along these lines:
{noformat}
if (no. of known map-output locations than no. of copier threads) {
try to schedule 75% of copies off-rack
try schedule 25% of copies in-rack
}
{noformat}
This could lead to better utilization of both in-rack switch b/w...
Clearly we want to schedule more cross-switch than in-rack since off-rack
copies will take significantly more time; hence the 75-25 split.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5164) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS
[ https://issues.apache.org/jira/browse/MAPREDUCE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nemon lou updated MAPREDUCE-5164: - Status: Patch Available (was: Open) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS - Key: MAPREDUCE-5164 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5164 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: nemon lou Assignee: nemon lou Attachments: MAPREDUCE-5164.patch HADOOP_CLIENT_OPTS does not take effect when type mapred job -list and mapred queue -list. The mapred script omit it -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5164) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS
[ https://issues.apache.org/jira/browse/MAPREDUCE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13636180#comment-13636180 ] nemon lou commented on MAPREDUCE-5164: -- No test case added as it is a script change. command mapred job and mapred queue omit HADOOP_CLIENT_OPTS - Key: MAPREDUCE-5164 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5164 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: nemon lou Assignee: nemon lou Attachments: MAPREDUCE-5164.patch HADOOP_CLIENT_OPTS does not take effect when type mapred job -list and mapred queue -list. The mapred script omit it -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5164) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS
nemon lou created MAPREDUCE-5164: Summary: command mapred job and mapred queue omit HADOOP_CLIENT_OPTS Key: MAPREDUCE-5164 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5164 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: nemon lou Assignee: nemon lou HADOOP_CLIENT_OPTS does not take effect when type mapred job -list and mapred queue -list. The mapred script omit it -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5164) command mapred job and mapred queue omit HADOOP_CLIENT_OPTS
[ https://issues.apache.org/jira/browse/MAPREDUCE-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nemon lou updated MAPREDUCE-5164: - Attachment: MAPREDUCE-5164.patch command mapred job and mapred queue omit HADOOP_CLIENT_OPTS - Key: MAPREDUCE-5164 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5164 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: nemon lou Assignee: nemon lou Attachments: MAPREDUCE-5164.patch HADOOP_CLIENT_OPTS does not take effect when type mapred job -list and mapred queue -list. The mapred script omit it -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5155) Race condition in test case TestFetchFailure cause it to fail
[
https://issues.apache.org/jira/browse/MAPREDUCE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634784#comment-13634784
]
nemon lou commented on MAPREDUCE-5155:
--
Another test case failed with the same reason:
Failed tests: testUpdatedNodes(org.apache.hadoop.mapreduce.v2.app.TestMRApp):
Expecting 2 more completion events for killed expected:4 but was:2
These lines of code has race condition issue:
{quote}
app.waitForState(task1Attempt, TaskAttemptState.KILLED);
app.waitForState(task2Attempt, TaskAttemptState.KILLED);
events = job.getTaskAttemptCompletionEvents(0, 100);
Assert.assertEquals(Expecting 2 more completion events for killed, 4,
events.length);
{quote}
Race condition in test case TestFetchFailure cause it to fail
-
Key: MAPREDUCE-5155
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5155
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.0.3-alpha
Environment: Suse x86_64 GNU/Linux
Java(TM) SE Runtime Environment (build 1.6.0_32-b05
Reporter: nemon lou
Priority: Minor
Attachments:
org.apache.hadoop.mapreduce.v2.app.TestFetchFailure-output.txt,
org.apache.hadoop.mapreduce.v2.app.TestFetchFailure.txt
I run into this once:
testFetchFailureWithRecovery(org.apache.hadoop.mapreduce.v2.app.TestFetchFailure):
Num completion events not correct expected:1 but was:0
There is a race condition between job.getTaskAttemptCompletionEvents and
dealing with JOB_TASK_ATTEMPT_COMPLETED event.
If job.getTaskAttemptCompletionEvents invoked because of task in SUCCEEDED
state ,but before JOB_TASK_ATTEMPT_COMPLETED event scheduled,the test case
will fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5155) Race condition in test case TestFetchFailure cause it to fail
nemon lou created MAPREDUCE-5155: Summary: Race condition in test case TestFetchFailure cause it to fail Key: MAPREDUCE-5155 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5155 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 2.0.3-alpha Environment: Suse x86_64 GNU/Linux Java(TM) SE Runtime Environment (build 1.6.0_32-b05 Reporter: nemon lou Priority: Minor I run into this once: testFetchFailureWithRecovery(org.apache.hadoop.mapreduce.v2.app.TestFetchFailure): Num completion events not correct expected:1 but was:0 There is a race condition between job.getTaskAttemptCompletionEvents and dealing with JOB_TASK_ATTEMPT_COMPLETED event.If job.getTaskAttemptCompletionEvents invoked because of task in SUCCEEDED state ,but before JOB_TASK_ATTEMPT_COMPLETED event scheduled,the test case will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5155) Race condition in test case TestFetchFailure cause it to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nemon lou updated MAPREDUCE-5155: - Attachment: org.apache.hadoop.mapreduce.v2.app.TestFetchFailure.txt org.apache.hadoop.mapreduce.v2.app.TestFetchFailure-output.txt Logs are uploaded Race condition in test case TestFetchFailure cause it to fail - Key: MAPREDUCE-5155 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5155 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 2.0.3-alpha Environment: Suse x86_64 GNU/Linux Java(TM) SE Runtime Environment (build 1.6.0_32-b05 Reporter: nemon lou Priority: Minor Attachments: org.apache.hadoop.mapreduce.v2.app.TestFetchFailure-output.txt, org.apache.hadoop.mapreduce.v2.app.TestFetchFailure.txt I run into this once: testFetchFailureWithRecovery(org.apache.hadoop.mapreduce.v2.app.TestFetchFailure): Num completion events not correct expected:1 but was:0 There is a race condition between job.getTaskAttemptCompletionEvents and dealing with JOB_TASK_ATTEMPT_COMPLETED event.If job.getTaskAttemptCompletionEvents invoked because of task in SUCCEEDED state ,but before JOB_TASK_ATTEMPT_COMPLETED event scheduled,the test case will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5155) Race condition in test case TestFetchFailure cause it to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nemon lou updated MAPREDUCE-5155: - Description: I run into this once: testFetchFailureWithRecovery(org.apache.hadoop.mapreduce.v2.app.TestFetchFailure): Num completion events not correct expected:1 but was:0 There is a race condition between job.getTaskAttemptCompletionEvents and dealing with JOB_TASK_ATTEMPT_COMPLETED event. If job.getTaskAttemptCompletionEvents invoked because of task in SUCCEEDED state ,but before JOB_TASK_ATTEMPT_COMPLETED event scheduled,the test case will fail. was: I run into this once: testFetchFailureWithRecovery(org.apache.hadoop.mapreduce.v2.app.TestFetchFailure): Num completion events not correct expected:1 but was:0 There is a race condition between job.getTaskAttemptCompletionEvents and dealing with JOB_TASK_ATTEMPT_COMPLETED event.If job.getTaskAttemptCompletionEvents invoked because of task in SUCCEEDED state ,but before JOB_TASK_ATTEMPT_COMPLETED event scheduled,the test case will fail. Race condition in test case TestFetchFailure cause it to fail - Key: MAPREDUCE-5155 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5155 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 2.0.3-alpha Environment: Suse x86_64 GNU/Linux Java(TM) SE Runtime Environment (build 1.6.0_32-b05 Reporter: nemon lou Priority: Minor Attachments: org.apache.hadoop.mapreduce.v2.app.TestFetchFailure-output.txt, org.apache.hadoop.mapreduce.v2.app.TestFetchFailure.txt I run into this once: testFetchFailureWithRecovery(org.apache.hadoop.mapreduce.v2.app.TestFetchFailure): Num completion events not correct expected:1 but was:0 There is a race condition between job.getTaskAttemptCompletionEvents and dealing with JOB_TASK_ATTEMPT_COMPLETED event. If job.getTaskAttemptCompletionEvents invoked because of task in SUCCEEDED state ,but before JOB_TASK_ATTEMPT_COMPLETED event scheduled,the test case will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4560) Job can get stuck in a deadlock between mappers and reducers for low values of mapreduce.job.reduce.slowstart.completedmaps (1)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435848#comment-13435848 ] nemon lou commented on MAPREDUCE-4560: -- Do you use the FIFO scheduler? If so ,have a look at MAPREDUCE-4299 Job can get stuck in a deadlock between mappers and reducers for low values of mapreduce.job.reduce.slowstart.completedmaps (1) - Key: MAPREDUCE-4560 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4560 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Rahul Jain Fix For: 2.0.0-alpha This issue has been seen with MapReduceV2, never with MapReduceV1 in our lab systems. The parameter mapreduce.job.reduce.slowstart.completedmaps=0.05 (the default value). We found Application master stuck in a deadlock between mappers and reducers with no progress in the job; the sequence appears to be: 1. Initial available map/reduce slots were allocated to mappers 2. Once mappers made progress and few of them completed, reducers started occupying few of the slots due to low values of above config param. 3. The scheduler appears to not give priority to mappers over reducers; after a while in our system we saw all slots occupied by reducers. 4. Since there were still mapper tasks not yet assigned any slot, the map phase never completed. 5. The system entered a deadlock state where reducers occupy all available slots, but are waiting for mappers to be complete; mappers cannot move forward because of no slot available. The workaround in our system was to set mapreduce.job.reduce.slowstart.completedmaps=1 and the issue was no longer seen. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4460) Refresh queue throws IO exception after configuring wrong queue capacity
[
https://issues.apache.org/jira/browse/MAPREDUCE-4460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13432985#comment-13432985
]
nemon lou commented on MAPREDUCE-4460:
--
The same to MAPREDUCE-3763.Any updates here?
Refresh queue throws IO exception after configuring wrong queue capacity
Key: MAPREDUCE-4460
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4460
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.3, 2.1.0-alpha
Reporter: Nishan Shetty
Assignee: Arun C Murthy
Priority: Critical
Scenario:
1.My setup has a,b queues(each with capacity say 50%) under root queue
2.Start the process
3.Add one more queue 'c' under root
4.Configure some capacity for 'c' such that total capacity of a,b,c is not
equal to 100
5.Now do refresh queues, it will throw exception as wrong capacity(This is
expected as capacity was not equal to 100).
6.Now reconfigure queue capacities of a,b,c such that total capacity is 100
5.Now do refresh queues again
Observed that it throws IO exception
{noformat}
java.io.IOException: Failed to re-init queues
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:216)
at
org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:174)
at
org.apache.hadoop.yarn.server.resourcemanager.api.impl.pb.service.RMAdminProtocolPBServiceImpl.refreshQueues(RMAdminProtocolPBServiceImpl.java:62)
at
org.apache.hadoop.yarn.proto.RMAdminProtocol$RMAdminProtocolService$2.callBlockingMethod(RMAdminProtocol.java:122)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:427)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:916)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1692)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1688)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1686)
Caused by: org.apache.hadoop.metrics2.MetricsException: Metrics source
QueueMetrics,q0=root,q1=c already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:126)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:107)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:216)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.forQueue(QueueMetrics.java:129)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics.forQueue(QueueMetrics.java:119)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.init(LeafQueue.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.parseQueue(CapacityScheduler.java:313)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.parseQueue(CapacityScheduler.java:328)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:246)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:213)
... 11 more
at LocalTrace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl:
Failed to re-init queues
at
org.apache.hadoop.yarn.factories.impl.pb.YarnRemoteExceptionFactoryPBImpl.createYarnRemoteException(YarnRemoteExceptionFactoryPBImpl.java:50)
at
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:40)
at
org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:184)
at
org.apache.hadoop.yarn.server.resourcemanager.api.impl.pb.service.RMAdminProtocolPBServiceImpl.refreshQueues(RMAdminProtocolPBServiceImpl.java:62)
at
org.apache.hadoop.yarn.proto.RMAdminProtocol$RMAdminProtocolService$2.callBlockingMethod(RMAdminProtocol.java:122)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:427)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:916)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1692)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1688)
at
