[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615370#comment-13615370 ] Hadoop QA commented on HBASE-7904: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12575709/7904-v9.txt against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 6 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:red}-1 site{color}. The patch appears to cause mvn site goal to fail. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.io.encoding.TestChangingEncoding Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5028//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5028//console This message is automatically generated. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, > 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, > 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, > 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, > hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617050#comment-13617050 ] Ted Yu commented on HBASE-7904: --- TestChangingEncoding test failure should be fixed by addendum to HBASE-8063. Patch v9 is ready for review: 7904-v9-hadoop-2.0.txt allows all mapreduce jobs to pass using hadoop 2.0 > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, > 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, > 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, > 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, > hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617092#comment-13617092 ] stack commented on HBASE-7904: -- Why this change? - this.coprocessorHost = new RegionCoprocessorHost(this, rsServices, conf); + this.coprocessorHost = new RegionCoprocessorHost(this, rsServices, baseConf); This seems to be only real addition: +HBaseConfiguration.merge(this.conf, jobConf); -- is that right? Did something change in the hadoop2 snapshot? A yarn patch got committed? Why we have to do this now? -GenericOptionsParser opts = new GenericOptionsParser(new Configuration(cluster.getConfiguration()), args); +GenericOptionsParser opts = new GenericOptionsParser(new Configuration( + cluster.getConfiguration()), args); Configuration conf = opts.getConfiguration(); +// copy or add the necessary configuration values from the map reduce config to the hbase config +copyConfigurationValues(UTIL.getConfiguration(), conf); Why create a new HBaseConfiguration instance? What is wrong w/ the cluster instance? How does cluster config relate to UTIL.getConfiguration? (Are they the same?) > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, > 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, > 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, > 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, > hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617363#comment-13617363 ] Ted Yu commented on HBASE-7904: --- bq. Why this change? When SecureBulkLoadEndpoint tries to access hdfs, org.apache.hadoop.ipc.RPC.setProtocolEngine() would call Configuration.setClass(). Without the change, we would get: {code} 2013-03-25 22:36:19,493 ERROR [IPC Server handler 2 on 40900] access.SecureBulkLoadEndpoint$1(240): Failed to complete bulk load java.lang.UnsupportedOperationException: Immutable Configuration at org.apache.hadoop.hbase.CompoundConfiguration.setClass(CompoundConfiguration.java:474) at org.apache.hadoop.ipc.RPC.setProtocolEngine(RPC.java:193) {code} bq. This seems to be only real addition I followed your suggestion. Since debugging output from copyNewConfEntries() is no longer needed, I removed that method. bq. Did something change in the hadoop2 snapshot? I would have to go into recent checkins in more detail. This may be answered by a YARN developer. bq. Why we have to do this now? We have been doing that for the remaining tests. See one example from testSimpleCase(): {code} Configuration conf = opts.getConfiguration(); // copy or add the necessary configuration values from the map reduce config to the hbase config copyConfigurationValues(UTIL.getConfiguration(), conf); args = opts.getRemainingArgs(); {code} bq. Why create a new HBaseConfiguration instance? I don't see reference to HBaseConfiguration in TestImportExport. The patch doesn't introduce HBaseConfiguration either. bq. How does cluster config relate to UTIL.getConfiguration? cluster is a MiniHBaseCluster instance. UTIL is an HBaseTestingUtility where we store all yarn-related conf parameters. So they're not the same. copyConfigurationValues() passes all yarn-related conf parameters from UTIL.getConfiguration() to conf. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, > 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, > 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, > 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, > hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617686#comment-13617686 ] stack commented on HBASE-7904: -- bq. I would have to go into recent checkins in more detail. This may be answered by a YARN developer. I suppose I could go look myself but I ask because supposedly there was an incompatibility issue. If tests are now passing, is it because the snapshot has incompatibilities fixed now? (I suppose I could look for myself but you are supposed to know what is going on here since you are 'fixing' it). Or is it just this config. manipulation that was needed and there is no compatibility problem. Would think we would want to answer the above before we had confidence in the commit. bq. I don't see reference to HBaseConfiguration in TestImportExport. The patch doesn't introduce HBaseConfiguration either. UTIL.getConfiguration is an HBaseConfiguration. bq. cluster is a MiniHBaseCluster instance. UTIL is an HBaseTestingUtility where we store all yarn-related conf parameters. Where do you think cluster gets its config from? Isn't it using UTIL.getConfiguration? I was asking questions about the config juggling because to me it looked like the test was doing a bunch of config copying to no useful end; that a little thought might make it so there was less juggling going on. Would help the developer who comes along after trying to make sense of this test. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, > 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, > 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, > 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, > hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617700#comment-13617700 ] Ted Yu commented on HBASE-7904: --- I looked at the changes in hadoop-2 branch in the past week. There were fixes to CapacityScheduler but I think they may not be directly related to the test failures we saw. bq. Or is it just this config. manipulation that was needed I think so. bq. Where do you think cluster gets its config from? Isn't it using UTIL.getConfiguration? We obtain cluster at the beginning of the test: {code} public static void beforeClass() throws Exception { cluster = UTIL.startMiniCluster(); {code} Config manipulation happens in UTIL.startMiniMapReduceCluster(). But startMiniCluster() only calls startMiniDFSCluster(), startMiniZKCluster() and startMiniHBaseCluster(), in that order. This is why a little extra manipulation is needed in the tests of TestImportExport. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, > 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, > 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, > 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, > hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617841#comment-13617841 ] stack commented on HBASE-7904: -- I don't follow at all what you are saying above. You are not sure what fixed the problem and you present a spaghetti list of how we start up clusters in our test util to conclude "a little extra manipulation is needed...". +1 on patch. I want to put this issue out of its misery. Can open a new one if still incompatibility issues running against hadoop2. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, > 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, > 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, > 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, > hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617844#comment-13617844 ] Ted Yu commented on HBASE-7904: --- bq. But startMiniCluster() only calls startMiniDFSCluster(), startMiniZKCluster() and startMiniHBaseCluster(), in that order. I was just saying that cluster (a MiniHBaseCluster) wouldn't get its config from UTIL.getConfiguration because the effective config is updated in UTIL.startMiniMapReduceCluster(). bq. Can open a new one if still incompatibility issues running against hadoop2. True. Another JIRA would be needed when 2.0.4-ALPHA is released. In the mean time, if we see mapreduce errors using 2.0 SNAPSHOT build, we can provide feedback upstream. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, > 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, > 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, > 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, > hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617958#comment-13617958 ] Ted Yu commented on HBASE-7904: --- Integrated patch v9 to trunk. This would use 2.0.4-SNAPSHOT for build on hadoop 2. Let's see what test failures we get on the next HBase-TRUNK-on-Hadoop-2.0.0 build. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, > 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, > 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, > 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, > hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617993#comment-13617993 ] Hudson commented on HBASE-7904: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #469 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/469/]) HBASE-7904 Make mapreduce jobs pass based on 2.0.4-alpha (Revision 1462686) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java * /hbase/trunk/pom.xml > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618127#comment-13618127 ] Hudson commented on HBASE-7904: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #470 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/470/]) HBASE-7904 Addendum turns off vmem check (Revision 1462701) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618220#comment-13618220 ] Ted Yu commented on HBASE-7904: --- >From trunk build results on Apache and EC2 Jenkins, mapreduce jobs passed. Integrated to 0.95 as well. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618247#comment-13618247 ] Hudson commented on HBASE-7904: --- Integrated in hbase-0.95 #115 (See [https://builds.apache.org/job/hbase-0.95/115/]) HBASE-7904 Make mapreduce jobs pass based on 2.0.4-alpha (Ted Yu) (Revision 1462873) Result = SUCCESS tedyu : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java * /hbase/branches/0.95/pom.xml > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618532#comment-13618532 ] Hudson commented on HBASE-7904: --- Integrated in hbase-0.95-on-hadoop2 #49 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/49/]) HBASE-7904 Make mapreduce jobs pass based on 2.0.4-alpha (Ted Yu) (Revision 1462873) Result = FAILURE tedyu : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java * /hbase/branches/0.95/pom.xml > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619155#comment-13619155 ] Andrew Purtell commented on HBASE-7904: --- Does this change mean that coprocessors will have a different view of configuration than the HRegion itself? I.e. not see updates to configuration via CompoundConfiguration from the schema? {noformat} Index: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java === --- hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java (revision 1461235) +++ hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java (working copy) @@ -484,7 +484,7 @@ this.rsAccounting = this.rsServices.getRegionServerAccounting(); // don't initialize coprocessors if not running within a regionserver // TODO: revisit if coprocessors should load in other cases - this.coprocessorHost = new RegionCoprocessorHost(this, rsServices, conf); + this.coprocessorHost = new RegionCoprocessorHost(this, rsServices, baseConf); this.metricsRegionWrapper = new MetricsRegionWrapperImpl(this); this.metricsRegion = new MetricsRegion(this.metricsRegionWrapper); } else { {noformat} > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619158#comment-13619158 ] Andrew Purtell commented on HBASE-7904: --- Wait, I just saw the v9 patch has been committed to trunk and 0.95. Why is this JIRA still open? See my above comment. Should this be addressed as an addendum here or as a new JIRA? > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619159#comment-13619159 ] Siddharth Seth commented on HBASE-7904: --- Posted a summary at https://issues.apache.org/jira/browse/YARN-449?focusedCommentId=13619145&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13619145 bq. Did something change in the hadoop2 snapshot? A yarn patch got committed? Nothing that should've affected HBase (given the previous comment about clusters not running in parallel) bq. I suppose I could go look myself but I ask because supposedly there was an incompatibility issue. Do you happen to know what this incompatibility was (and against which version) ? bq. Regarding TestImportExport - I'm not sure why this test is using the HBaseTestingUtility differently than other tests. It was missing setting YARN conf in certain places. bq. We just want something that worked like minimr in hadoop1. We start the cluster do our tests, not in //, then shut down the cluster. I misunderstood you when you said // above. Yes we run our tests in //. We do not set up a minimr cluster once and run tests in // against it. I'm still not clear about this. (not so relevant now that the tests are passing, but would be nice to know). " Yes we run our tests in //" - this is a little confusing. As an example, can TestTableMapReduce run in parallel with TestHFileOutputFormat ?, or can multiple tests within TestTableMapReduce run in parallel, or something else. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619162#comment-13619162 ] Andrew Purtell commented on HBASE-7904: --- Since I am the "owner" of coprocessors and this patch changes how RegionCoprocessorHost is instantiated, I would really have appreciated a ping. Thanks. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619175#comment-13619175 ] Ted Yu commented on HBASE-7904: --- bq. Why is this JIRA still open? 0.95 should not ship with 2.0.4-SNAPSHOT build. That was why I left this open. If this JIRA is resolved, we can open another JIRA to update pom.xml when 2.0.4-ALPHA is released. bq. this patch changes how RegionCoprocessorHost is instantiated I exchanged a few emails with Nicolas who was the author of CompoundConfiguration. My initial plan was to allow CompoundConfiguration.setClass() to be called. He advised against that approach because CompoundConfiguration was designed to be immutable. I understand that the patch changed behavior for RegionCoprocessorHost instantiation. If there is a better approach to solving the issue of running mapreduce job on hadoop 2.0 *and* maintaining immutability of CompoundConfiguration, I would be more than happy to adopt. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619180#comment-13619180 ] Andrew Purtell commented on HBASE-7904: --- bq. I understand that the patch changed behavior for RegionCoprocessorHost instantiation. So you are ok with coprocessors having a different configuration than HRegion? Don't you forsee issues with that? > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619182#comment-13619182 ] Andrew Purtell commented on HBASE-7904: --- bq. 0.95 should not ship with 2.0.4-SNAPSHOT build. That was why I left this open. I think it is confusing, because the patch on this issue has already been committed to the target branches. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619186#comment-13619186 ] Andrew Purtell commented on HBASE-7904: --- bq. I exchanged a few emails with Nicolas who was the author of CompoundConfiguration. All interactions should be on JIRA or on the dev@ or user@ email lists. Decisions made in private communication are against the Apache Way. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619229#comment-13619229 ] stack commented on HBASE-7904: -- bq. Do you happen to know what this incompatibility was (and against which version) ? [~sseth] No. This issue started out as an incompability issue (see original [~ted_yu] comments at head of issue) and then ended w/ some configuration manipulations that seem gratiuitous. I ask the author what fixed the issue and I get that they "think" the config. manipulations are all that is necessary. I have spent a bunch of time trying to follow what is going on here, but gave up, and +1'd it as a means of putting the issue behind us since the patch seemed relatively harmless. bq. As an example, can TestTableMapReduce run in parallel with TestHFileOutputFormat ?, or can multiple tests within TestTableMapReduce run in parallel, or something else. Yes, hbase tests can be run in parallel: e.g. TestTableMapReduce and TestHFileOutputFormat run at same time. Our surefire runs our test suite by firing off N JVMs each running a distinct, long-running test. What we do not do is set up a mini mr cluster and then run many unit tests against. bq. I exchanged a few emails with Nicolas who was the author of CompoundConfiguration. [~ted_yu] And you say nothing in here about it? Why do it in private? They call it open source for a reason. Suggest we revert this patch. Seems broke that CPs see another config. because "Chane to HRegion to not use a 'CompoundConfiguration' when talking to HDFS, due to the inability to change the RPC engine with CompoundConfiguration. (Causes several *SecureLoad tests to hang)" according to https://issues.apache.org/jira/browse/YARN-449 (a conclusion that is absent here). > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619234#comment-13619234 ] Ted Yu commented on HBASE-7904: --- I tried the following change: {code} Index: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java === --- hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java (revision 1463235) +++ hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java (working copy) @@ -484,7 +484,7 @@ this.rsAccounting = this.rsServices.getRegionServerAccounting(); // don't initialize coprocessors if not running within a regionserver // TODO: revisit if coprocessors should load in other cases - this.coprocessorHost = new RegionCoprocessorHost(this, rsServices, baseConf); + this.coprocessorHost = new RegionCoprocessorHost(this, rsServices, conf); this.metricsRegionWrapper = new MetricsRegionWrapperImpl(this); this.metricsRegion = new MetricsRegion(this.metricsRegionWrapper); } else { Index: hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java === --- hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java (revision 1463235) +++ hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java (working copy) @@ -121,7 +121,7 @@ baseStagingDir = SecureBulkLoadUtil.getBaseStagingDir(conf); try { - fs = FileSystem.get(conf); + fs = FileSystem.get(new Configuration(conf)); fs.mkdirs(baseStagingDir, PERM_HIDDEN); fs.setPermission(baseStagingDir, PERM_HIDDEN); //no sticky bit in hadoop-1.0, making directory nonempty so it never gets erased {code} But I still got: {code} 2013-04-01 14:43:38,337 ERROR [IPC Server handler 0 on 52052] access.SecureBulkLoadEndpoint$1(240): Failed to complete bulk load java.lang.UnsupportedOperationException: Immutable Configuration at org.apache.hadoop.hbase.CompoundConfiguration.setClass(CompoundConfiguration.java:474) at org.apache.hadoop.ipc.RPC.setProtocolEngine(RPC.java:193) at org.apache.hadoop.hdfs.NameNodeProxies.createNNProxyWithClientProtocol(NameNodeProxies.java:249) at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:168) at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:129) at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:421) at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:388) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:126) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2312) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:87) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2346) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2328) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:352) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:164) at org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint$1.run(SecureBulkLoadEndpoint.java:224) at org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint$1.run(SecureBulkLoadEndpoint.java:218) {code} I also tried this: {code} Index: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java === --- hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java (revision 1463235) +++ hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java (working copy) @@ -484,7 +484,7 @@ this.rsAccounting = this.rsServices.getRegionServerAccounting(); // don't initialize coprocessors if not running within a regionserver // TODO: revisit if coprocessors should load in other cases - this.coprocessorHost = new RegionCoprocessorHost(this, rsServices, baseConf); + this.coprocessorHost = new RegionCoprocessorHost(this, rsServices, conf); this.metricsRegionWrapper = new MetricsRegionWrapperImpl(this); this.metricsRegion = new MetricsRegion(this.metricsRegionWrapper); } else { Index: hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java === --- hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java (revision 1463235) +++ hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java (working copy) @@ -31,6 +31,7 @@ import org.apache.hadoop.fs.permission.FsPermission; import org.apache.hadoop.hbase.Coprocessor; import
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619238#comment-13619238 ] Andrew Purtell commented on HBASE-7904: --- bq. Suggest we revert this patch. Seems broke that CPs see another config. +1 for revert > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619240#comment-13619240 ] Ted Yu commented on HBASE-7904: --- bq. Suggest we revert this patch I agree. We should consider a better design first. What about the notification that checkin to 0.95 is on hold ? > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619246#comment-13619246 ] Ted Yu commented on HBASE-7904: --- Another solution I can think of is to expose two Configurations from CoprocessorEnvironment: one that is immutable, the other mutable. The second one would be used for this call: {code} fs = FileSystem.get(conf); {code} BTW, here is javadoc for FileSystem.get(): {code} * Get a filesystem instance based on the uri, the passed * configuration and the user * @param uri of the filesystem * @param conf the configuration to use {code} I don't see mentioning that conf cannot be immutable. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619251#comment-13619251 ] stack commented on HBASE-7904: -- [~ted_yu] Please back this out of trunk and 0.95 (I tried but cannot figure what has been committed w/o doing a bunch of work) > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619252#comment-13619252 ] Andrew Purtell commented on HBASE-7904: --- bq. Another solution I can think of is to expose two Configurations from CoprocessorEnvironment: one that is immutable, the other mutable. The second one would be used for this call I think that's closer to a solution. I think what you want is to take the immutable CompoundConfiguration and create a new mutable Configuration with it to pass to the RegionCoprocessorHost constructor, one initialized with the overrides provided by the compound view. But I suggest reverting this patch and tackling the issues here with focused JIRAs. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619259#comment-13619259 ] Ted Yu commented on HBASE-7904: --- Backed out of 0.95 and trunk. Will attach patch v10 soon. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619261#comment-13619261 ] Andrew Purtell commented on HBASE-7904: --- I'm going to close this issue. Let's deal with the individual items here with JIRAs that focus on them individually. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619284#comment-13619284 ] Ted Yu commented on HBASE-7904: --- YARN-449, marked Resolved, is linked to this issue. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619286#comment-13619286 ] Andrew Purtell commented on HBASE-7904: --- Unlinked from YARN-449. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619354#comment-13619354 ] Ted Yu commented on HBASE-7904: --- I plan to introduce the following methods to CoprocessorEnvironment and : {code} /** @return the mutable configuration */ public Configuration getMutableConfiguration(); /** @return the immutable configuration */ public Configuration getImmutableConfiguration(); {code} w.r.t. the existing method: {code} /** @return the configuration */ public Configuration getConfiguration(); {code} We can either remove it or let it return the immutable config (current behavior). > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619356#comment-13619356 ] Andrew Purtell commented on HBASE-7904: --- {quote} I plan to introduce the following methods to CoprocessorEnvironment and : {noformat} /** @return the mutable configuration */ public Configuration getMutableConfiguration(); /** @return the immutable configuration */ public Configuration getImmutableConfiguration(); {noformat} {quote} -1, see my above comment. I know you read it, why are you ignoring it? -1, this issue is closed. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619363#comment-13619363 ] Ted Yu commented on HBASE-7904: --- I think you were referring to https://issues.apache.org/jira/browse/HBASE-7904?focusedCommentId=13619252&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13619252 SecureBulkLoadEndpoint needs a way to get access to both Configuration's, right ? See the following in SecureBulkLoadEndpoint#start(): {code} conf = env.getConfiguration(); {code} > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619366#comment-13619366 ] Andrew Purtell commented on HBASE-7904: --- -1 I won't discuss this further here, this issue is closed. Kindly open a JIRA to address the SecureBulkLoadEndpoint issue. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619367#comment-13619367 ] Ted Yu commented on HBASE-7904: --- @Andy: Maybe you're clear about the new JIRA (title, description, etc) If you can open the JIRA, that would be great. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619370#comment-13619370 ] Andrew Purtell commented on HBASE-7904: --- bq. If you can open the JIRA, that would be great. Surely this is an April Fool's joke and not a display of petulance. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619438#comment-13619438 ] Hudson commented on HBASE-7904: --- Integrated in HBase-TRUNK #4005 (See [https://builds.apache.org/job/HBase-TRUNK/4005/]) HBASE-7904 Revert, due to concern over coprocessor exposure (Revision 1463326) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java * /hbase/trunk/pom.xml > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619881#comment-13619881 ] Ted Yu commented on HBASE-7904: --- >From >https://issues.apache.org/jira/browse/HBASE-7904?focusedCommentId=13610919&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13610919 > , this JIRA was blocker for 0.95 Is that no longer the case ? > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620462#comment-13620462 ] Siddharth Seth commented on HBASE-7904: --- bq. This issue started out as an incompability issue (see original Ted Yu comments at head of issue) If this was about MiniMRCluster being deprecated, the replacement is MiniMRClientClusterFactory which exists in branch-1 as well. Don't think this is an incompatible change. If using the new interface, MiniMRClientCluster.getConfig() can be used instead of MiniMRCluster.createJobConf(). bq. Yes, hbase tests can be run in parallel: e.g. TestTableMapReduce and TestHFileOutputFormat run at same time. Our surefire runs our test suite by firing off N JVMs each running a distinct, long-running test. Alright. So no parallel tests against a single cluster, but multiple clusters can run at the same time since these two test classes spin up their own clusters. MAPREDUCE-5083 becomes important to prevent the clusters clobbering each other. Will pull that into the 2.0.4-alpha branch. The summary posted over at YARN-449 (with these additions) should be enough to get the hbase mapreduce tests passing. I'm guessing the HBase specific changes will eventually make it in via separate jiras. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13621198#comment-13621198 ] stack commented on HBASE-7904: -- [~sseth] bq. Will pull that into the 2.0.4-alpha branch. Thanks. I added an ask to MAPREDUCE-5083. I could commit it but sounds like you are more on it... thanks for taking care of us. I made HBASE-8258 for making hbase-side changes. > Make mapreduce jobs pass based on 2.0.4-alpha > - > > Key: HBASE-7904 > URL: https://issues.apache.org/jira/browse/HBASE-7904 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Ted Yu >Priority: Critical > Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, > 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, > 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, > 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, > 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt > > > 2.0.3-alpha has been released. > We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira