[jira] [Created] (MAPREDUCE-3891) NM should handle cleaning up containers when it shuts down ( and kill containers from an earlier instance when it comes back up after an unclean shutdown )
NM should handle cleaning up containers when it shuts down ( and kill containers from an earlier instance when it comes back up after an unclean shutdown ) --- Key: MAPREDUCE-3891 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3891 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah Ideally, the NM should wait for a limited amount of time when it gets a shutdown signal for existing containers to complete and kill the containers ( if we pick an aggressive approach ) after this time interval. For NMs which come up after an unclean shutdown, the NM should look through its directories for existing container.pids and try and kill an existing containers matching the pids found. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits - Key: MAPREDUCE-3796 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3796 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
Change MiniYarnCluster to escape special chars in testname -- Key: MAPREDUCE-3775 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor When using MiniYarnCluster with the testname set to a nested classname, the $ within the class name creates issues with the container launch scripts as they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting
FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting Key: MAPREDUCE-3765 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3765 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor FifoScheduler uses default min 1 GB regardless of the configuration value set for minimum memory allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3739) Document all missing default configuration values in the relevant *default.xml files.
Document all missing default configuration values in the relevant *default.xml files. - Key: MAPREDUCE-3739 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3739 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Fix For: 0.23.1, 0.24.0 There seem to be quite a few configuration settings that are used in the code but missing from the *default.xml files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3523) Exclude mrapp-generated-classpath from published jars
Exclude mrapp-generated-classpath from published jars - Key: MAPREDUCE-3523 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3523 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Critical Fix For: 0.23.1, 0.24.0 Downstream projects will be generating their own classpath file to allow yarn to launch containers correctly in a unit-test env which may potentially create a clash if 2 versions are present. The current code in MRApps assumes that the generated classpath is always present which may need to be addressed if this file is not present when deployed on a cluster. An option for a fix could be to pass the location of the classpath file via the MiniYarnCluster constructor and propagate that to MRApps via some form of configuration value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3485) DISKS_FAILED -101 error code should be defined in same location as ABORTED_CONTAINER_EXIT_STATUS
DISKS_FAILED -101 error code should be defined in same location as ABORTED_CONTAINER_EXIT_STATUS Key: MAPREDUCE-3485 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3485 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Fix For: 0.23.1 With MAPREDUCE-3121, it is defined in ContainerExecutor as part of yarn-nodemanager which would be a problem for client-side code if it needs to understand the exit code. A short term fix would be to move it into YarnConfiguration where ABORTED_CONTAINER_EXIT_STATUS is defined. A longer term fix would be to find a more formal and extensible approach for new yarn framework error codes to be added and be easily accessible to client-side code or other AMs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3465) org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin fails on 0.23
org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin fails on 0.23 Key: MAPREDUCE-3465 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3465 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.23.1 Running org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.121 sec FAILURE! Tests in error: testParsingProcStatAndCpuFile(org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin): /homes/hortonhi/dev/hadoop-common/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/CPUINFO_943711651 (No such file or directory) testParsingProcMemFile(org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin): /homes/hortonhi/dev/hadoop-common/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/MEMINFO_943711651 (No such file or directory) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3444) trunk/0.23 builds broken
trunk/0.23 builds broken - Key: MAPREDUCE-3444 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3444 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Fix For: 0.23.1 https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/208/ https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1310/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3434) Nightly build broken
Nightly build broken - Key: MAPREDUCE-3434 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3434 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Priority: Blocker Fix For: 0.23.1 https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk/901/ Results : Failed tests: testSleepJob(org.apache.hadoop.mapreduce.v2.TestMRJobs) testRandomWriter(org.apache.hadoop.mapreduce.v2.TestMRJobs) testDistributedCache(org.apache.hadoop.mapreduce.v2.TestMRJobs) Tests in error: org.apache.hadoop.mapreduce.v2.TestMROldApiJobs: Failed to Start org.apache.hadoop.mapreduce.v2.TestMROldApiJobs org.apache.hadoop.mapreduce.v2.TestUberAM: Failed to Start org.apache.hadoop.mapreduce.v2.TestMRJobs Likely due to either of: MAPREDUCE-3415. improve MiniMRYarnCluster DistributedShell JAR resolution (tucu) MAPREDUCE-3169. Create a new MiniMRCluster equivalent which only provides client APIs cross MR1 and MR2. (Ahmed via tucu) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3425) uber job hangs and fails to get killed properly on a job kill signal
uber job hangs and fails to get killed properly on a job kill signal Key: MAPREDUCE-3425 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3425 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0, 0.23.1 Reporter: Hitesh Shah Fix For: 0.24.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3426) uber-jobs tried to write outputs into wrong dir
uber-jobs tried to write outputs into wrong dir --- Key: MAPREDUCE-3426 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3426 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Fix For: 0.23.1 Incorrect setup of the uber tasks causes tasks to try to write intermidiate outputs into dirs that the user does not have permissions to write to on a secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3420) Make uber jobs functional
Make uber jobs functional - Key: MAPREDUCE-3420 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3420 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.1 Umbrella jira for getting uber jobs to work correctly with YARN/MRv2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3421) Error in MR AM shutdown when running an uber-job
Error in MR AM shutdown when running an uber-job Key: MAPREDUCE-3421 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3421 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Fix For: 0.23.1 Exception thrown during the un-registration process in the ContainerAllocator#stop. Exception trace in next comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3407) Wrong jar getting used in TestMR*Jobs* for MiniMRYarnCluster
Wrong jar getting used in TestMR*Jobs* for MiniMRYarnCluster Key: MAPREDUCE-3407 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3407 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.23.1 pom for mapreduce-client-jobclient sets system property to incorrect jar name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3332) contrib/raid compile breaks due to changes in hdfs/protocol/datatransfer/Sender#writeBlock related to checksum handling
contrib/raid compile breaks due to changes in hdfs/protocol/datatransfer/Sender#writeBlock related to checksum handling Key: MAPREDUCE-3332 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3332 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/raid Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial Fix For: 0.23.1 [javac] /Users/Hitesh/dev/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java:783: writeBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock,org.apache.hadoop.security.token.Tokenorg.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier,java.lang.String,org.apache.hadoop.hdfs.protocol.DatanodeInfo[],org.apache.hadoop.hdfs.protocol.DatanodeInfo,org.apache.hadoop.hdfs.protocol.datatransfer.BlockConstructionStage,int,long,long,long,org.apache.hadoop.util.DataChecksum) in org.apache.hadoop.hdfs.protocol.datatransfer.Sender cannot be applied to (org.apache.hadoop.hdfs.protocol.ExtendedBlock,org.apache.hadoop.security.token.Tokenorg.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier,java.lang.String,org.apache.hadoop.hdfs.protocol.DatanodeInfo[],nulltype,org.apache.hadoop.hdfs.protocol.datatransfer.BlockConstructionStage,int,long,long,long) [javac] new Sender(out).writeBlock(block.getBlock(), block.getBlockToken(), , [javac]^ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3318) TestSeveral timing out in commit builds
TestSeveral timing out in commit builds --- Key: MAPREDUCE-3318 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3318 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3321) Disable some failing legacy tests for MRv2 builds to go through
Disable some failing legacy tests for MRv2 builds to go through --- Key: MAPREDUCE-3321 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3321 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Attachments: MR-3321.1.patch By-product of MR-3214. Disable tests for the short term until fixes are available for all tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3314) Support AM specific blacklisting of nodes on the RM
Support AM specific blacklisting of nodes on the RM --- Key: MAPREDUCE-3314 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3314 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.1 The AM blacklists certain nodes due to task failures but the RM does not know of this and can allocate blacklisted nodes to the AM again and again. RM should support a protocol that allows AM to inform the RM of nodes that it should not allocate to it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3262) Handle missing events in failure scenarios
Handle missing events in failure scenarios -- Key: MAPREDUCE-3262 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3262 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.0 Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3213) AM should kill all running tasks on a node when it gets marked as blacklisted
AM should kill all running tasks on a node when it gets marked as blacklisted - Key: MAPREDUCE-3213 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3213 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3214) ant mapreduce tests failing
ant mapreduce tests failing --- Key: MAPREDUCE-3214 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3214 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.0 Umbrella jira for various test failures -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3215) org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk
org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk -- Key: MAPREDUCE-3215 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3215 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testNoJobSetupCleanup took 13.271 sec FAILED Number of part-files is 0 and not 1 junit.framework.AssertionFailedError: Number of part-files is 0 and not 1 at org.apache.hadoop.mapreduce.TestNoJobSetupCleanup.submitAndValidateJob(TestNoJobSetupCleanup.java:60) at org.apache.hadoop.mapreduce.TestNoJobSetupCleanup.testNoJobSetupCleanup(TestNoJobSetupCleanup.java:70) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3216) ant test TestNoDefaultsJobConf fails on trunk
ant test TestNoDefaultsJobConf fails on trunk - Key: MAPREDUCE-3216 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3216 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testNoDefaults took 4.703 sec Caused an ERROR Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses. java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses. at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:118) at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:81) at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:74) at org.apache.hadoop.mapred.JobClient.init(JobClient.java:460) at org.apache.hadoop.mapred.JobClient.init(JobClient.java:439) at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:809) at org.apache.hadoop.conf.TestNoDefaultsJobConf.testNoDefaults(TestNoDefaultsJobConf.java:83) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3217) ant test TestAuditLogger fails on trunk
ant test TestAuditLogger fails on trunk --- Key: MAPREDUCE-3217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3217 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testKeyValLogFormat took 0.096 sec Testcase: testAuditLoggerWithoutIP took 0.005 sec Testcase: testAuditLoggerWithIP took 0.417 sec Caused an ERROR java.io.IOException: Unknown protocol: org.apache.hadoop.ipc.TestRPC$TestProtocol at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:615) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) java.io.IOException: java.io.IOException: Unknown protocol: org.apache.hadoop.ipc.TestRPC$TestProtocol at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:615) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) at org.apache.hadoop.ipc.Client.call(Client.java:1085) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:244) at $Proxy6.ping(Unknown Source) at org.apache.hadoop.mapred.TestAuditLogger.testAuditLoggerWithIP(TestAuditLogger.java:150) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3218) ant test TestTokenCache failing on trunk
ant test TestTokenCache failing on trunk Key: MAPREDUCE-3218 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3218 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testTokenCache took 11.607 sec Testcase: testLocalJobTokenCache took 12.224 sec Testcase: testGetTokensForNamenodes took 0.009 sec Testcase: testGetTokensForHftpFS took 0.676 sec Testcase: testGetJTPrincipal took 0.023 sec FAILED Failed to substitute HOSTNAME_PATTERN with hostName expected:jt/foo@BAR but was:null junit.framework.AssertionFailedError: Failed to substitute HOSTNAME_PATTERN with hostName expected:jt/foo@BAR but was:null at org.apache.hadoop.mapreduce.security.TestTokenCache.testGetJTPrincipal(TestTokenCache.java:392) Testcase: testGetTokensForViewFS took 0.019 sec -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3219) ant test TestDelegationToken failing on trunk
ant test TestDelegationToken failing on trunk - Key: MAPREDUCE-3219 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3219 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testDelegationToken took 2.043 sec Caused an ERROR Client Hitesh tries to renew a token with renewer specified as alice at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:239) at org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:4829) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:632) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) org.apache.hadoop.security.AccessControlException: Client Hitesh tries to renew a token with renewer specified as alice at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:239) at org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:4829) at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:632) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57) at org.apache.hadoop.mapreduce.Cluster.renewDelegationToken(Cluster.java:399) at org.apache.hadoop.mapred.JobClient$Renewer.renew(JobClient.java:475) at org.apache.hadoop.security.token.Token.renew(Token.java:310) at org.apache.hadoop.mapred.JobClient.renewDelegationToken(JobClient.java:1088) at org.apache.hadoop.mapreduce.security.token.delegation.TestDelegationToken.testDelegationToken(TestDelegationToken.java:89) Caused by: org.apache.hadoop.security.AccessControlException: Client Hitesh tries to renew a token with renewer specified as alice at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:239) at org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:4829) at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:632) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) at org.apache.hadoop.ipc.Client.call(Client.java:1085) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:244) at $Proxy11.renewDelegationToken(Unknown Source) at org.apache.hadoop.mapreduce.Cluster.renewDelegationToken(Cluster.java:397) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3220) ant test TestCombineOutputCollector failing on trunk
ant test TestCombineOutputCollector failing on trunk Key: MAPREDUCE-3220 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3220 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testsuite: org.apache.hadoop.mapred.TestCombineOutputCollector Tests run: 2, Failures: 2, Errors: 0, Time elapsed: 1.591 sec Testcase: testCustomCollect took 0.363 sec FAILED taskReporter.progress(); Never wanted here: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) But invoked here: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) junit.framework.AssertionFailedError: taskReporter.progress(); Never wanted here: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) But invoked here: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) at org.apache.hadoop.mapred.TestCombineOutputCollector.testCustomCollect(TestCombineOutputCollector.java:118) Testcase: testDefaultCollect took 1.211 sec FAILED taskReporter.progress(); Wanted 1 time: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) But was 1 times. Undesired invocation: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) junit.framework.AssertionFailedError: taskReporter.progress(); Wanted 1 time: - at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) But was 1 times. Undesired invocation: - at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1202) at org.apache.hadoop.mapred.TestCombineOutputCollector.testDefaultCollect(TestCombineOutputCollector.java:139) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3221) ant test TestSubmitJob failing on trunk
ant test TestSubmitJob failing on trunk --- Key: MAPREDUCE-3221 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3221 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testJobWithInvalidMemoryReqs took 2.588 sec Testcase: testSecureJobExecution took 4.089 sec FAILED java.io.IOException: org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol org.apache.hadoop.hdfs.protocol.ClientProtocol version mismatch. (client = 69, server = 70) at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:617) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) junit.framework.AssertionFailedError: java.io.IOException: org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol org.apache.hadoop.hdfs.protocol.ClientProtocol version mismatch. (client = 69, server = 70) at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:617) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1517) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1513) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1511) at org.apache.hadoop.mapred.TestSubmitJob.testSecureJobExecution(TestSubmitJob.java:270) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3222) ant test TestTaskContext failing on trunk
ant test TestTaskContext failing on trunk - Key: MAPREDUCE-3222 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3222 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testContextStatus took 29.977 sec FAILED null expected:map[ sort] but was:map[] junit.framework.ComparisonFailure: null expected:map[ sort] but was:map[] at org.apache.hadoop.mapreduce.TestTaskContext.testContextStatus(TestTaskContext.java:120) Testcase: testMapContextProgress took 17.371 sec Testcase: testReduceContextProgress took 16.267 sec -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3207) TestMRCLI failing on trunk
TestMRCLI failing on trunk Key: MAPREDUCE-3207 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3207 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Failing tests: 7: Archive: Deleting a file in archive 8: Archive: Renaming a file in archive -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3198) Change mode for hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/mock-container-executor to 755
Change mode for hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/mock-container-executor to 755 - Key: MAPREDUCE-3198 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3198 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Trivial Fix For: 0.23.0 The file is checked in with 644 permissions. TestLinuxContainerExecutorWithMocks changes the file mode to add executable permission if needed resulting in a modified file for 'git/svn status' when tests are run. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3183) hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml missing license header
hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml missing license header Key: MAPREDUCE-3183 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3183 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3158) Fix trunk build failures
Fix trunk build failures Key: MAPREDUCE-3158 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3158 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Fix For: 0.23.0 https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Commit/1060/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3155) Document and likewise implement relevant checks/escape functions for what form of input is acceptable for setting up a container launch context i.e. special chars in
Document and likewise implement relevant checks/escape functions for what form of input is acceptable for setting up a container launch context i.e. special chars in resource names, env vars and the commands --- Key: MAPREDUCE-3155 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3155 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.0 What should a user of yarn escape/not escape when passing in input for the container launch contexts - localized resources' names are used to create symlinks. - Are special chars supported in symlinks or do they need to be escaped? - Likewise for environment variables and commands. Current implementation uses a shell script to setup the environment and launch the commands. What should the user be aware of when setting up the launch context? The input also should be such that a user should not need to change code based on what platform or flavor of shell is being used to setup the env and run the commands. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3137) Fix broken merge of MR-2719 to 0.23 branch for the distributed shell test case
Fix broken merge of MR-2719 to 0.23 branch for the distributed shell test case --- Key: MAPREDUCE-3137 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3137 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial Fix For: 0.23.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3098) Report Application status as well as ApplicationMaster status in GetApplicationReportResponse
Report Application status as well as ApplicationMaster status in GetApplicationReportResponse -- Key: MAPREDUCE-3098 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3098 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.0 Currently, an application report received by the client from the RM/ASM for a given application returns the status of the application master. It does not return the status of the application i.e. whether that particular job succeeded or failed. The AM status would be one of FINISHED (SUCCEEDED should be renamed to FINISHED as AM state does not indicate overall success/failure), FAILED or KILLED. The final state sent by the AM to the RM in the FinishApplicationMasterRequest should be exposed to the client as ApplicationState. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
