MAPREDUCE 6099
Hi, I have added a new api to org.apache.hadoop.mapreduce.lib.input.CombinFileInputFormat getSplits(JobContext, List). Link to the JIRA : https://issues.apache.org/jira/browse/MAPREDUCE-6099 Could anyone please review the patch ? It is kind of a blocker for a patch to hive that I need to submit. Thanks, Pankit
[jira] [Resolved] (MAPREDUCE-6114) TestMRCJCFileInputFormat#testAddInputPath fails in trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe resolved MAPREDUCE-6114. --- Resolution: Duplicate Dup of MAPREDUCE-6094. > TestMRCJCFileInputFormat#testAddInputPath fails in trunk > > > Key: MAPREDUCE-6114 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6114 > Project: Hadoop Map/Reduce > Issue Type: Test >Reporter: Ted Yu >Priority: Minor > > This can be reproduced locally: > {code} > Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.474 sec <<< > FAILURE! - in org.apache.hadoop.mapreduce.lib.input.TestMRCJCFileInputFormat > testAddInputPath(org.apache.hadoop.mapreduce.lib.input.TestMRCJCFileInputFormat) > Time elapsed: 0.86 sec <<< ERROR! > java.io.IOException: No FileSystem for scheme: s3 > at > org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2583) > at > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2590) > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) > at > org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2629) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2611) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169) > at > org.apache.hadoop.mapreduce.lib.input.TestMRCJCFileInputFormat.testAddInputPath(TestMRCJCFileInputFormat.java:55) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6115) TestPipeApplication#testSubmitter fails in trunk
Ted Yu created MAPREDUCE-6115: - Summary: TestPipeApplication#testSubmitter fails in trunk Key: MAPREDUCE-6115 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6115 Project: Hadoop Map/Reduce Issue Type: Test Reporter: Ted Yu Priority: Minor This can be reproduced locally: {code} Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.89 sec <<< FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication testSubmitter(org.apache.hadoop.mapred.pipes.TestPipeApplication) Time elapsed: 0.061 sec <<< FAILURE! java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.apache.hadoop.mapred.pipes.TestPipeApplication.testSubmitter(TestPipeApplication.java:310) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6114) TestMRCJCFileInputFormat fails in trunk
Ted Yu created MAPREDUCE-6114: - Summary: TestMRCJCFileInputFormat fails in trunk Key: MAPREDUCE-6114 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6114 Project: Hadoop Map/Reduce Issue Type: Test Reporter: Ted Yu Priority: Minor This can be reproduced locally: {code} Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.474 sec <<< FAILURE! - in org.apache.hadoop.mapreduce.lib.input.TestMRCJCFileInputFormat testAddInputPath(org.apache.hadoop.mapreduce.lib.input.TestMRCJCFileInputFormat) Time elapsed: 0.86 sec <<< ERROR! java.io.IOException: No FileSystem for scheme: s3 at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2583) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2590) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2629) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2611) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169) at org.apache.hadoop.mapreduce.lib.input.TestMRCJCFileInputFormat.testAddInputPath(TestMRCJCFileInputFormat.java:55) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Hadoop-Mapreduce-trunk - Build # 1911 - Still Failing
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1911/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 28871 lines...] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.602 sec - in org.apache.hadoop.mapred.TestSequenceFileAsBinaryOutputFormat Results : Failed tests: TestJavaSerialization.testMapReduceJob:127 expected:<[a ]1> but was:<[0 1]1> TestPipeApplication.testSubmitter:310 null Tests in error: TestMRCJCFileInputFormat.testAddInputPath:55 ? IO No FileSystem for scheme: s3 Tests run: 508, Failures: 2, Errors: 1, Skipped: 11 [INFO] [INFO] Reactor Summary: [INFO] [INFO] hadoop-mapreduce-client ... SUCCESS [ 2.615 s] [INFO] hadoop-mapreduce-client-core .. SUCCESS [ 57.485 s] [INFO] hadoop-mapreduce-client-common SUCCESS [ 27.163 s] [INFO] hadoop-mapreduce-client-shuffle ... SUCCESS [ 4.412 s] [INFO] hadoop-mapreduce-client-app ... SUCCESS [08:45 min] [INFO] hadoop-mapreduce-client-hs SUCCESS [05:11 min] [INFO] hadoop-mapreduce-client-jobclient . FAILURE [ 01:32 h] [INFO] hadoop-mapreduce-client-hs-plugins SKIPPED [INFO] hadoop-mapreduce-client-nativetask SKIPPED [INFO] Apache Hadoop MapReduce Examples .. SKIPPED [INFO] hadoop-mapreduce .. SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 01:48 h [INFO] Finished at: 2014-09-29T15:11:10+00:00 [INFO] Final Memory: 41M/600M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on project hadoop-mapreduce-client-jobclient: There are test failures. [ERROR] [ERROR] Please refer to /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :hadoop-mapreduce-client-jobclient Build step 'Execute shell' marked build as failure [FINDBUGS] Skipping publisher since build result is FAILURE Archiving artifacts Sending artifact delta relative to Hadoop-Mapreduce-trunk #1870 Archived 2 artifacts Archive block size is 32768 Received 0 blocks and 20207052 bytes Compression is 0.0% Took 14 sec Updating HDFS-7093 Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
[jira] [Created] (MAPREDUCE-6113) An ability to override LocalResourceVisibility for public resources
Andrey Stepachev created MAPREDUCE-6113: --- Summary: An ability to override LocalResourceVisibility for public resources Key: MAPREDUCE-6113 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6113 Project: Hadoop Map/Reduce Issue Type: Improvement Components: client, distributed-cache Reporter: Andrey Stepachev Assignee: Andrey Stepachev Priority: Minor Currently it is impossible to specify application level for files in distributed cache in YARN. For applications with big common data it is very crucial to have ability to share data only for application and not spam public cache. Moreover, public localiser is limited for parallel downloads, so applications with big caches affect application startup of other applications. Current logic is complicated and based on file permissions. Making proposed changes, user can override public level to any level (PUBLIC/APP/PRIVATE) without breaking anything, and even not need to bother with permission, if private cache is needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)