See <https://builds.apache.org/job/HBase-TRUNK-security/171/changes>
Changes: [mbautin] [jira] [HBASE-5684] Make ProcessBasedLocalHBaseCluster run HDFS and make it more robust Summary: Currently ProcessBasedLocalHBaseCluster runs on top of raw local filesystem. We need it to start a process-based HDFS cluster as well. We also need to make the whole thing more stable so we can use it in unit tests. Also all logs of local HBase cluster daemons are now tailed to the primary log of the unit test with the appropriate prefixes to make debugging easier. This is a trunk diff. The 89-fb version is D2709. Test Plan: Run the new unit test multiple times (10x or 50x). Run all unit tests. Reviewers: tedyu, stack, lhofhansl, nspiegelberg, amirshim, JIRA Reviewed By: tedyu Differential Revision: https://reviews.facebook.net/D2757 [larsh] HBASE-5604 Addendum. Remove test as per discussion on jira. [stack] HBASE-5747 Forward port "hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test" [stack] HBASE-5443 Convert the client protocol of HRegionInterface to PB ------------------------------------------ [...truncated 8139 lines...] Running org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.877 sec Running org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.573 sec Running org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.334 sec Running org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.856 sec Running org.apache.hadoop.hbase.mapred.TestTableInputFormat Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.169 sec Running org.apache.hadoop.hbase.TestGlobalMemStoreSize Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.037 sec Running org.apache.hadoop.hbase.mapred.TestTableMapReduce Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 85.693 sec Running org.apache.hadoop.hbase.mapreduce.TestHLogRecordReader Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.726 sec Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 112.674 sec Running org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 104.11 sec Running org.apache.hadoop.hbase.mapreduce.TestWALPlayer Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.512 sec Running org.apache.hadoop.hbase.mapreduce.TestImportTsv Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 82.253 sec Running org.apache.hadoop.hbase.regionserver.wal.TestLogRolling Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 575.218 sec Running org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 221.093 sec Running org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 41.893 sec Running org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.261 sec Running org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 113.162 sec Running org.apache.hadoop.hbase.mapreduce.TestImportExport Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.073 sec Running org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.907 sec Running org.apache.hadoop.hbase.catalog.TestMetaReaderEditorNoCluster Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.203 sec Running org.apache.hadoop.hbase.catalog.TestMetaReaderEditor Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.642 sec Running org.apache.hadoop.hbase.TestAcidGuarantees Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 96.981 sec Running org.apache.hadoop.hbase.catalog.TestCatalogTracker Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 5.47 sec Running org.apache.hadoop.hbase.client.TestFromClientSide3 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.35 sec Running org.apache.hadoop.hbase.client.TestTimestampsFilter Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.522 sec Running org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.092 sec Running org.apache.hadoop.hbase.client.TestShell Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 97.628 sec Running org.apache.hadoop.hbase.client.TestHCM Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.273 sec Running org.apache.hadoop.hbase.client.TestFromClientSide Tests run: 56, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 215.879 sec Running org.apache.hadoop.hbase.client.TestMetaScanner Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.179 sec Running org.apache.hadoop.hbase.client.TestHTableUtil Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.231 sec Running org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 559.701 sec Running org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.168 sec Running org.apache.hadoop.hbase.client.TestHTablePool$TestHTableReusablePool Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.14 sec Running org.apache.hadoop.hbase.client.TestHTablePool$TestHTableThreadLocalPool Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.752 sec Running org.apache.hadoop.hbase.client.replication.TestReplicationAdmin Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.885 sec Running org.apache.hadoop.hbase.client.TestMultipleTimestamps Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 29.128 sec Running org.apache.hadoop.hbase.client.TestScannerTimeout Tests run: 4, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 196.929 sec <<< FAILURE! Running org.apache.hadoop.hbase.metrics.TestMetricsMBeanBase Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.117 sec Running org.apache.hadoop.hbase.client.TestMultiParallel Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.426 sec Running org.apache.hadoop.hbase.constraint.TestConstraint Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.088 sec Running org.apache.hadoop.hbase.rest.TestGzipFilter Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.032 sec Running org.apache.hadoop.hbase.rest.TestMultiRowResource Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.463 sec Running org.apache.hadoop.hbase.rest.TestScannersWithFilters Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.552 sec Running org.apache.hadoop.hbase.rest.client.TestRemoteAdmin Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.495 sec Running org.apache.hadoop.hbase.rest.client.TestRemoteTable Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.733 sec Running org.apache.hadoop.hbase.client.TestAdmin Tests run: 36, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 101.887 sec <<< FAILURE! Running org.apache.hadoop.hbase.rest.TestVersionResource Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.685 sec Running org.apache.hadoop.hbase.rest.TestRowResource Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 54.54 sec <<< FAILURE! Running org.apache.hadoop.hbase.rest.TestStatusResource Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.311 sec Running org.apache.hadoop.hbase.rest.TestTableResource Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.745 sec Running org.apache.hadoop.hbase.rest.TestScannerResource Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 29.224 sec Running org.apache.hadoop.hbase.rest.TestSchemaResource Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.075 sec Running org.apache.hadoop.hbase.io.encoding.TestLoadAndSwitchEncodeOnDisk Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.183 sec Running org.apache.hadoop.hbase.thrift.TestThriftServer Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.833 sec Running org.apache.hadoop.hbase.io.encoding.TestDataBlockEncoders Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 109.895 sec Running org.apache.hadoop.hbase.io.encoding.TestUpgradeFromHFileV1ToEncoding Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 84.016 sec Running org.apache.hadoop.hbase.io.hfile.TestHFileSeek Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.909 sec Running org.apache.hadoop.hbase.io.encoding.TestChangingEncoding Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 240.488 sec Running org.apache.hadoop.hbase.io.hfile.slab.TestSlabCache Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.318 sec Running org.apache.hadoop.hbase.io.hfile.TestChecksum Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.913 sec Running org.apache.hadoop.hbase.io.hfile.TestHFileBlockIndex Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.065 sec Running org.apache.hadoop.hbase.io.hfile.TestLruBlockCache Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.789 sec Running org.apache.hadoop.hbase.io.hfile.slab.TestSingleSizeCache Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.173 sec Running org.apache.hadoop.hbase.io.hfile.TestFixedFileTrailer Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.628 sec Running org.apache.hadoop.hbase.io.hfile.TestHFileBlockCompatibility Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.16 sec Running org.apache.hadoop.hbase.io.hfile.TestHFilePerformance Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.609 sec Running org.apache.hadoop.hbase.io.hfile.TestHFileBlock Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.242 sec Running org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.93 sec Running org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite Tests run: 36, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.824 sec Running org.apache.hadoop.hbase.avro.TestAvroServer Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.937 sec Running org.apache.hadoop.hbase.replication.regionserver.TestReplicationSourceManager Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.084 sec Running org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.652 sec Running org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 456.426 sec Running org.apache.hadoop.hbase.replication.TestReplicationSource Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.744 sec Running org.apache.hadoop.hbase.replication.TestReplicationPeer Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.312 sec Running org.apache.hadoop.hbase.replication.TestMultiSlaveReplication Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 68.275 sec Running org.apache.hadoop.hbase.replication.TestReplicationZookeeper Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.664 sec Running org.apache.hadoop.hbase.ipc.TestDelayedRpc Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.309 sec Running org.apache.hadoop.hbase.TestZooKeeper Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.043 sec Running org.apache.hadoop.hbase.replication.TestMasterReplication Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 60.958 sec Running org.apache.hadoop.hbase.TestHBaseTestingUtility Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 60.028 sec Running org.apache.hadoop.hbase.replication.TestReplication Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 145.393 sec Results : Failed tests: testNoSuchCF(org.apache.hadoop.hbase.rest.TestRowResource): expected:<503> but was:<404> Tests in error: testQualifierAccess(org.apache.hadoop.hbase.security.access.TestAccessControlFilter): Failed after attempts=10, exceptions:(..) testRead(org.apache.hadoop.hbase.security.access.TestAccessController): Failed after attempts=10, exceptions:(..) testWrite(org.apache.hadoop.hbase.security.access.TestAccessController): Failed after attempts=10, exceptions:(..) testGrantRevoke(org.apache.hadoop.hbase.security.access.TestAccessController): Failed after attempts=10, exceptions:(..) testGrantRevokeAtQualifierLevel(org.apache.hadoop.hbase.security.access.TestAccessController): Failed after attempts=10, exceptions:(..) testCheckPermissions(org.apache.hadoop.hbase.security.access.TestAccessController): Failed after attempts=10, exceptions:(..) test2481(org.apache.hadoop.hbase.client.TestScannerTimeout): Failed after attempts=10, exceptions:(..) test2772(org.apache.hadoop.hbase.client.TestScannerTimeout): Failed after attempts=10, exceptions:(..) test3686a(org.apache.hadoop.hbase.client.TestScannerTimeout): Failed after attempts=10, exceptions:(..) test3686b(org.apache.hadoop.hbase.client.TestScannerTimeout): Failed after attempts=10, exceptions:(..) testForceSplit(org.apache.hadoop.hbase.client.TestAdmin): org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=6, exceptions:(..) testForceSplitMultiFamily(org.apache.hadoop.hbase.client.TestAdmin): org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=6, exceptions:(..) Tests run: 944, Failures: 1, Errors: 12, Skipped: 9 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 52:43.760s [INFO] Finished at: Sat Apr 14 05:57:52 UTC 2012 [INFO] Final Memory: 47M/578M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test (secondPartTestsExecution) on project hbase: There are test failures. [ERROR] [ERROR] Please refer to <https://builds.apache.org/job/HBase-TRUNK-security/ws/trunk/target/surefire-reports> for the individual test results. [ERROR] -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test (secondPartTestsExecution) on project hbase: There are test failures. Please refer to <https://builds.apache.org/job/HBase-TRUNK-security/ws/trunk/target/surefire-reports> for the individual test results. at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.MojoExecutor.executeForkedExecutions(MojoExecutor.java:365) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:199) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59) at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196) at org.apache.maven.cli.MavenCli.main(MavenCli.java:141) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352) Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures. Please refer to <https://builds.apache.org/job/HBase-TRUNK-security/ws/trunk/target/surefire-reports> for the individual test results. at org.apache.maven.plugin.surefire.SurefireHelper.reportExecution(SurefireHelper.java:87) at org.apache.maven.plugin.surefire.SurefirePlugin.writeSummary(SurefirePlugin.java:651) at org.apache.maven.plugin.surefire.SurefirePlugin.handleSummary(SurefirePlugin.java:625) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:137) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:98) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209) ... 23 more [ERROR] [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException Build step 'Invoke top-level Maven targets' marked build as failure Archiving artifacts Updating HBASE-5747 Updating HBASE-5684 Updating HBASE-5443 Updating HBASE-5708 Updating HBASE-5604 Recording test results
