I put up a new RC JMS. It still has flakies (though Duo fixed TestFromClientSide...). Was thinking that we could release beta-1 though it has flakies. We'll keep working on cutting these down as we approach GA. St.Ack
On Sun, Jan 7, 2018 at 10:02 PM, Stack <st...@duboce.net> wrote: > On Sun, Jan 7, 2018 at 3:14 AM, Jean-Marc Spaggiari < > jean-m...@spaggiari.org> wrote: > >> Ok, thanks Stack. I will keep it running all day long until I get a >> successful one. Is that useful that I report all the failed? Or just a >> wast >> of time? Here is the last failed: >> >> [INFO] Results: >> [INFO] >> [ERROR] Failures: >> [ERROR] TestFromClientSide.testCheckAndDeleteWithCompareOp:4982 >> expected:<false> but was:<true> >> [ERROR] Errors: >> [ERROR] TestDLSAsyncFSWAL>AbstractTestDLS.testThreeRSAbort:401 » >> TableNotFound Region ... >> [INFO] >> [ERROR] Tests run: 3585, Failures: 1, Errors: 1, Skipped: 44 >> [INFO] >> >> >> > Thanks for bringing up flakies. If we look at the nightlies' run, we can > get the current list. Probably no harm if all tests pass once in a while > (smile). > > Looking at your findings, TestFromClientSide.testCheckAndDeleteWithCompareOp > looks to be new to beta-1. Its a cranky one. I'm looking at it. Might punt > to beta-2 if can't figure it by tomorrow. HBASE-19731. > > TestDLSAsyncFSWAL is a flakey that unfortunately passes locally. > > Let me see what others we have... > > S > > > > > > > > >> JMS >> >> 2018-01-07 1:55 GMT-05:00 Apekshit Sharma <a...@cloudera.com>: >> >> > bq. Don't you think we have enough branches already mighty Appy? >> > Yeah we do...sigh. >> > >> > >> > idk about that. But don't we need a *patch* branch branch-2.0 (just like >> > branch-1.4) where we "make backwards-compatible bug fixes" and a *minor* >> > branch branch-2 where we "add functionality in a backwards-compatible >> > manner". >> > Quotes are from http://hbase.apache.org/book.h >> tml#hbase.versioning.post10. >> > I stumbled on this issue when thinking about backporting >> > https://issues.apache.org/jira/browse/HBASE-17436 for 2.1. >> > >> > -- Appy >> > >> > >> > On Sat, Jan 6, 2018 at 4:11 PM, stack <saint....@gmail.com> wrote: >> > >> > > It is not you. There are a bunch of flies we need to fix. This >> latter is >> > > for sure flakey. Let me take a look. Thanks, JMS. >> > > >> > > S >> > > >> > > On Jan 6, 2018 5:57 PM, "Jean-Marc Spaggiari" < >> jean-m...@spaggiari.org> >> > > wrote: >> > > >> > > I might not doing the right magic to get that run.... If someone is >> able >> > to >> > > get all the tests pass, can you please share the command you run? >> > > >> > > Thanks, >> > > >> > > JMS >> > > >> > > >> > > [INFO] Results: >> > > [INFO] >> > > [ERROR] Failures: >> > > [ERROR] TestFromClientSide.testCheckAndDeleteWithCompareOp:4982 >> > > expected:<false> but was:<true> >> > > [ERROR] >> > > org.apache.hadoop.hbase.master.assignment.TestMergeTableRegi >> onsProcedure >> > > .testMergeRegionsConcurrently(org.apache.hadoop.hbase.master.assig >> > > nment.TestMergeTableRegionsProcedure) >> > > [ERROR] Run 1: >> > > TestMergeTableRegionsProcedure.setup:111->resetProcExecutorT >> estingKillFl >> > > ag:138 >> > > expected executor to be running >> > > [ERROR] Run 2: >> > > TestMergeTableRegionsProcedure.tearDown:128-> >> > > resetProcExecutorTestingKillFl >> > > ag:138 >> > > expected executor to be running >> > > [INFO] >> > > [ERROR] >> > > org.apache.hadoop.hbase.master.assignment.TestMergeTableRegi >> onsProcedure >> > > .testMergeTwoRegions(org.apache.hadoop.hbase.master.assignment.Tes >> > > tMergeTableRegionsProcedure) >> > > [ERROR] Run 1: >> > > TestMergeTableRegionsProcedure.setup:111->resetProcExecutorT >> estingKillFl >> > > ag:138 >> > > expected executor to be running >> > > [ERROR] Run 2: >> > > TestMergeTableRegionsProcedure.tearDown:128-> >> > > resetProcExecutorTestingKillFl >> > > ag:138 >> > > expected executor to be running >> > > [INFO] >> > > [ERROR] >> > > org.apache.hadoop.hbase.master.assignment.TestMergeTableRegi >> onsProcedure >> > . >> > > testRecoveryAndDoubleExecution(org.apache.hadoop.hbase.master.ass >> > > ignment.TestMergeTableRegionsProcedure) >> > > [ERROR] Run 1: >> > > TestMergeTableRegionsProcedure.setup:111->resetProcExecutorT >> estingKillFl >> > > ag:138 >> > > expected executor to be running >> > > [ERROR] Run 2: >> > > TestMergeTableRegionsProcedure.tearDown:128-> >> > > resetProcExecutorTestingKillFl >> > > ag:138 >> > > expected executor to be running >> > > [INFO] >> > > [ERROR] >> > > org.apache.hadoop.hbase.master.assignment.TestMergeTableRegi >> onsProcedure >> > . >> > > testRollbackAndDoubleExecution(org.apache.hadoop.hbase.master.ass >> > > ignment.TestMergeTableRegionsProcedure) >> > > [ERROR] Run 1: >> > > TestMergeTableRegionsProcedure.testRollbackAndDoubleExecution:272 >> > > expected:<true> but was:<false> >> > > [ERROR] Run 2: >> > > TestMergeTableRegionsProcedure.tearDown:128-> >> > > resetProcExecutorTestingKillFl >> > > ag:138 >> > > expected executor to be running >> > > [INFO] >> > > [ERROR] TestSnapshotQuotaObserverChore.testSnapshotSize:276 Waiting >> > > timed >> > > out after [30 000] msec >> > > [ERROR] >> > > TestHRegionWithInMemoryFlush>TestHRegion.testWritesWhileScan >> ning:3813 >> > > expected null, but was:<org.apache.hadoop.hbase. >> > NotServingRegionException: >> > > testWritesWhileScanning,,1515277468063.468265483817cb6da6320 >> 26ba5b306f6. >> > > is >> > > closing> >> > > [ERROR] Errors: >> > > [ERROR] TestDLSAsyncFSWAL>AbstractTestDLS.testThreeRSAbort:401 » >> > > TableNotFound testThr... >> > > [ERROR] >> > > org.apache.hadoop.hbase.master.balancer.TestRegionsOnMasterOptions. >> > > testRegionsOnAllServers(org.apache.hadoop.hbase.master.balancer. >> > > TestRegionsOnMasterOptions) >> > > [ERROR] Run 1: >> > > TestRegionsOnMasterOptions.testRegionsOnAllServers:94-> >> > > checkBalance:207->Object.wait:-2 >> > > » TestTimedOut >> > > [ERROR] Run 2: TestRegionsOnMasterOptions.testRegionsOnAllServers » >> > > Appears to be stuck in t... >> > > [INFO] >> > > [INFO] >> > > [ERROR] Tests run: 3604, Failures: 7, Errors: 2, Skipped: 44 >> > > [INFO] >> > > >> > > >> > > 2018-01-06 15:52 GMT-05:00 Jean-Marc Spaggiari < >> jean-m...@spaggiari.org >> > >: >> > > >> > > > Deleted the class to get all the tests running. Was running on the >> RC1 >> > > > from the tar. >> > > > >> > > > I know get those one failing. >> > > > >> > > > [ERROR] Failures: >> > > > [ERROR] TestFavoredStochasticLoadBalan >> cer.test2FavoredNodesDead:352 >> > > > Balancer did not run >> > > > [ERROR] TestRegionMergeTransactionOnCl >> uster.testCleanMergeReference: >> > > 284 >> > > > hdfs://localhost:45311/user/jmspaggi/test-data/7c269e83- >> > > > 5982-449e-8cf8-6babaaaa4c7c/data/default/testCleanMergeReference/ >> > > > f1bdc6441b090dbacb391c74eaf0d1d0 >> > > > [ERROR] Errors: >> > > > [ERROR] TestDLSAsyncFSWAL>AbstractTestDLS.testThreeRSAbort:401 » >> > > > TableNotFound Region ... >> > > > [INFO] >> > > > [ERROR] Tests run: 3604, Failures: 2, Errors: 1, Skipped: 44 >> > > > >> > > > >> > > > I have not been able to get all the tests passed locally for a >> while :( >> > > > >> > > > JM >> > > > >> > > > 2018-01-06 15:05 GMT-05:00 Ted Yu <yuzhih...@gmail.com>: >> > > > >> > > >> Looks like you didn't include HBASE-19666 which would be in the >> next >> > RC. >> > > >> >> > > >> On Sat, Jan 6, 2018 at 10:52 AM, Jean-Marc Spaggiari < >> > > >> jean-m...@spaggiari.org> wrote: >> > > >> >> > > >> > Trying with a different command line (mvn test -P runAllTests >> > > >> > -Dsurefire.secondPartThreadCount=12 >> -Dtest.build.data.basedirector >> > > >> y=/ram4g >> > > >> > ) I get all those one failing. How are you able to get >> everything >> > > >> passed??? >> > > >> > >> > > >> > [INFO] Results: >> > > >> > [INFO] >> > > >> > [ERROR] Failures: >> > > >> > [ERROR] TestDefaultCompactSelection.testCompactionRatio:74-> >> > TestCom >> > > >> > pactionPolicy.compactEquals:182->TestCompactionPolicy. >> > > compactEquals:201 >> > > >> > expected:<[[4, 2, 1]]> but was:<[[]]> >> > > >> > [ERROR] TestDefaultCompactSelection. >> > testStuckStoreCompaction:145->T >> > > >> > estCompactionPolicy.compactEquals:182->TestCompactionPolicy. >> > > >> compactEquals:201 >> > > >> > expected:<[[]30, 30, 30]> but was:<[[99, 30, ]30, 30, 30]> >> > > >> > [INFO] >> > > >> > [ERROR] Tests run: 1235, Failures: 2, Errors: 0, Skipped: 4 >> > > >> > >> > > >> > Second run: >> > > >> > [INFO] Results: >> > > >> > [INFO] >> > > >> > [ERROR] Failures: >> > > >> > [ERROR] TestDefaultCompactSelection.testCompactionRatio:74-> >> > > >> > TestCompactionPolicy.compactEquals:182->TestCompactionPolicy >> > > >> .compactEquals:201 >> > > >> > expected:<[[4, 2, 1]]> but was:<[[]]> >> > > >> > [ERROR] TestDefaultCompactSelection. >> > testStuckStoreCompaction:145-> >> > > >> > TestCompactionPolicy.compactEquals:182->TestCompactionPolicy >> > > >> .compactEquals:201 >> > > >> > expected:<[[]30, 30, 30]> but was:<[[99, 30, ]30, 30, 30]> >> > > >> > [INFO] >> > > >> > [ERROR] Tests run: 1235, Failures: 2, Errors: 0, Skipped: 4 >> > > >> > >> > > >> > Then again: >> > > >> > >> > > >> > [INFO] Results: >> > > >> > [INFO] >> > > >> > [ERROR] Failures: >> > > >> > [ERROR] TestDefaultCompactSelection.testCompactionRatio:74-> >> > > >> > TestCompactionPolicy.compactEquals:182->TestCompactionPolicy >> > > >> .compactEquals:201 >> > > >> > expected:<[[4, 2, 1]]> but was:<[[]]> >> > > >> > [ERROR] TestDefaultCompactSelection. >> > testStuckStoreCompaction:145-> >> > > >> > TestCompactionPolicy.compactEquals:182->TestCompactionPolicy >> > > >> .compactEquals:201 >> > > >> > expected:<[[]30, 30, 30]> but was:<[[99, 30, ]30, 30, 30]> >> > > >> > [INFO] >> > > >> > [ERROR] Tests run: 1235, Failures: 2, Errors: 0, Skipped: 4 >> > > >> > [INFO] >> > > >> > [INFO] ------------------------------ >> ------------------------------ >> > > >> > ------------ >> > > >> > [INFO] Reactor Summary: >> > > >> > >> > > >> > >> > > >> > Sound like it's always the exact same result. Do I have a way to >> > > exclude >> > > >> > this TestCompactionPolicy test from the run? >> > > >> > >> > > >> > Here are more details from the last failure: >> > > >> > ------------------------------------------------------------ >> > > >> > ------------------- >> > > >> > Test set: org.apache.hadoop.hbase.regionserver. >> > > TestDefaultCompactSelec >> > > >> tion >> > > >> > ------------------------------------------------------------ >> > > >> > ------------------- >> > > >> > Tests run: 4, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: >> > 1.323 >> > > s >> > > >> > <<< FAILURE! - in org.apache.hadoop.hbase.regionserver. >> > > >> > TestDefaultCompactSelection >> > > >> > testStuckStoreCompaction(org.apache.hadoop.hbase.regionserve >> > > >> r.TestDefaultCompactSelection) >> > > >> > Time elapsed: 1.047 s <<< FAILURE! >> > > >> > org.junit.ComparisonFailure: expected:<[[]30, 30, 30]> but >> > was:<[[99, >> > > >> 30, >> > > >> > ]30, 30, 30]> >> > > >> > at org.apache.hadoop.hbase.regionserver. >> > > >> > TestDefaultCompactSelection.testStuckStoreCompaction( >> > > >> > TestDefaultCompactSelection.java:145) >> > > >> > >> > > >> > testCompactionRatio(org.apache.hadoop.hbase.regionserver.Tes >> > > >> tDefaultCompactSelection) >> > > >> > Time elapsed: 0.096 s <<< FAILURE! >> > > >> > org.junit.ComparisonFailure: expected:<[[4, 2, 1]]> but >> was:<[[]]> >> > > >> > at org.apache.hadoop.hbase.regionserver. >> > > >> > TestDefaultCompactSelection.testCompactionRatio( >> > > >> > TestDefaultCompactSelection.java:74) >> > > >> > >> > > >> > >> > > >> > 2018-01-06 12:53:53,240 WARN [StoreOpener-22ce1d683ba4b6b93 >> > > >> 73a3c541ebab2a2-1] >> > > >> > util.CommonFSUtils(536): FileSystem doesn't support >> > setStoragePolicy; >> > > >> > HDFS-6584, HDFS-9345 not available. This is normal and expected >> on >> > > >> earlier >> > > >> > Hadoop versions. >> > > >> > java.lang.NoSuchMethodException: org.apache.hadoop.fs. >> > > LocalFileSystem. >> > > >> > setStoragePolicy(org.apache.hadoop.fs.Path, java.lang.String) >> > > >> > at java.lang.Class.getDeclaredMethod(Class.java:2130) >> > > >> > at org.apache.hadoop.hbase.util.CommonFSUtils. >> > > >> > invokeSetStoragePolicy(CommonFSUtils.java:528) >> > > >> > at org.apache.hadoop.hbase.util.CommonFSUtils. >> > > setStoragePolicy( >> > > >> > CommonFSUtils.java:518) >> > > >> > at org.apache.hadoop.hbase.region >> server.HRegionFileSystem. >> > > >> > setStoragePolicy(HRegionFileSystem.java:193) >> > > >> > at org.apache.hadoop.hbase.regionserver.HStore.<init>( >> > > >> > HStore.java:250) >> > > >> > at org.apache.hadoop.hbase.regionserver.HRegion. >> > > >> > instantiateHStore(HRegion.java:5497) >> > > >> > at org.apache.hadoop.hbase.regionserver.HRegion$1.call( >> > > >> > HRegion.java:1002) >> > > >> > at org.apache.hadoop.hbase.regionserver.HRegion$1.call( >> > > >> > HRegion.java:999) >> > > >> > at java.util.concurrent.FutureTas >> k.run(FutureTask.java:266) >> > > >> > at java.util.concurrent.Executors$RunnableAdapter. >> > > >> > call(Executors.java:511) >> > > >> > at java.util.concurrent.FutureTas >> k.run(FutureTask.java:266) >> > > >> > at java.util.concurrent.ThreadPoolExecutor.runWorker( >> > > >> > ThreadPoolExecutor.java:1149) >> > > >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run( >> > > >> > ThreadPoolExecutor.java:624) >> > > >> > at java.lang.Thread.run(Thread.java:748) >> > > >> > >> > > >> > >> > > >> > >> > > >> > 2018-01-06 12:53:53,322 DEBUG [main] util.CommonFSUtils(538): >> > > FileSystem >> > > >> > doesn't support setStoragePolicy; HDFS-6584, HDFS-9345 not >> > available. >> > > >> This >> > > >> > is normal and expected on earlier Hadoop versions. >> > > >> > java.lang.NoSuchMethodException: org.apache.hadoop.fs. >> > > LocalFileSystem. >> > > >> > setStoragePolicy(org.apache.hadoop.fs.Path, java.lang.String) >> > > >> > at java.lang.Class.getDeclaredMethod(Class.java:2130) >> > > >> > at org.apache.hadoop.hbase.util.CommonFSUtils. >> > > >> > invokeSetStoragePolicy(CommonFSUtils.java:528) >> > > >> > at org.apache.hadoop.hbase.util.CommonFSUtils. >> > > setStoragePolicy( >> > > >> > CommonFSUtils.java:518) >> > > >> > at org.apache.hadoop.hbase.region >> server.HRegionFileSystem. >> > > >> > setStoragePolicy(HRegionFileSystem.java:193) >> > > >> > at org.apache.hadoop.hbase.regionserver.HStore.<init>( >> > > >> > HStore.java:250) >> > > >> > at org.apache.hadoop.hbase.regionserver. >> > TestCompactionPolicy. >> > > >> > initialize(TestCompactionPolicy.java:109) >> > > >> > at org.apache.hadoop.hbase.regionserver. >> > > >> > TestCompactionPolicy.setUp(TestCompactionPolicy.java:69) >> > > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >> > > Method) >> > > >> > at sun.reflect.NativeMethodAccessorImpl.invoke( >> > > >> > NativeMethodAccessorImpl.java:62) >> > > >> > at sun.reflect.DelegatingMethodAccessorImpl.invoke( >> > > >> > DelegatingMethodAccessorImpl.java:43) >> > > >> > at java.lang.reflect.Method.invoke(Method.java:498) >> > > >> > at org.junit.runners.model.FrameworkMethod$1. >> > > runReflectiveCall( >> > > >> > FrameworkMethod.java:50) >> > > >> > at org.junit.internal.runners.mod >> el.ReflectiveCallable.run( >> > > >> > ReflectiveCallable.java:12) >> > > >> > at org.junit.runners.model.FrameworkMethod. >> > invokeExplosively( >> > > >> > FrameworkMethod.java:47) >> > > >> > at org.junit.internal.runners.statements.RunBefores. >> > > >> > evaluate(RunBefores.java:24) >> > > >> > at org.junit.internal.runners. >> > statements.RunAfters.evaluate( >> > > >> > RunAfters.java:27) >> > > >> > at org.junit.runners.ParentRunner.runLeaf( >> > > ParentRunner.java:325 >> > > >> ) >> > > >> > at org.junit.runners.BlockJUnit4ClassRunner.runChild( >> > > >> > BlockJUnit4ClassRunner.java:78) >> > > >> > at org.junit.runners.BlockJUnit4ClassRunner.runChild( >> > > >> > BlockJUnit4ClassRunner.java:57) >> > > >> > at org.junit.runners.ParentRunner$3.run( >> > > ParentRunner.java:290) >> > > >> > at org.junit.runners.ParentRunner$1.schedule( >> > > ParentRunner.java: >> > > >> 71) >> > > >> > at org.junit.runners.ParentRunner.runChildren( >> > > >> > ParentRunner.java:288) >> > > >> > at org.junit.runners.ParentRunner.access$000( >> > > ParentRunner.java: >> > > >> 58) >> > > >> > at org.junit.runners.ParentRunner$2.evaluate( >> > > >> > ParentRunner.java:268) >> > > >> > at org.junit.runners.ParentRunner.run(ParentRunner. >> > java:363) >> > > >> > at org.junit.runners.Suite.runChild(Suite.java:128) >> > > >> > at org.junit.runners.Suite.runChild(Suite.java:27) >> > > >> > at org.junit.runners.ParentRunner$3.run( >> > > ParentRunner.java:290) >> > > >> > at org.junit.runners.ParentRunner$1.schedule( >> > > ParentRunner.java: >> > > >> 71) >> > > >> > at org.junit.runners.ParentRunner.runChildren( >> > > >> > ParentRunner.java:288) >> > > >> > at org.junit.runners.ParentRunner.access$000( >> > > ParentRunner.java: >> > > >> 58) >> > > >> > at org.junit.runners.ParentRunner$2.evaluate( >> > > >> > ParentRunner.java:268) >> > > >> > at org.junit.runners.ParentRunner.run(ParentRunner. >> > java:363) >> > > >> > at org.apache.maven.surefire.junitcore.JUnitCore.run( >> > > >> > JUnitCore.java:55) >> > > >> > at org.apache.maven.surefire.junitcore.JUnitCoreWrapper. >> > > >> > createRequestAndRun(JUnitCoreWrapper.java:137) >> > > >> > at org.apache.maven.surefire.junitcore.JUnitCoreWrapper. >> > > >> > executeEager(JUnitCoreWrapper.java:107) >> > > >> > at org.apache.maven.surefire.junitcore.JUnitCoreWrapper. >> > > >> > execute(JUnitCoreWrapper.java:83) >> > > >> > at org.apache.maven.surefire.junitcore.JUnitCoreWrapper. >> > > >> > execute(JUnitCoreWrapper.java:75) >> > > >> > at org.apache.maven.surefire.juni >> tcore.JUnitCoreProvider. >> > > >> > invoke(JUnitCoreProvider.java:159) >> > > >> > at org.apache.maven.surefire.booter.ForkedBooter. >> > > >> > invokeProviderInSameClassLoader(ForkedBooter.java:373) >> > > >> > at org.apache.maven.surefire.booter.ForkedBooter. >> > > >> > runSuitesInProcess(ForkedBooter.java:334) >> > > >> > at org.apache.maven.surefire.boot >> er.ForkedBooter.execute( >> > > >> > ForkedBooter.java:119) >> > > >> > at org.apache.maven.surefire.booter.ForkedBooter.main( >> > > >> > ForkedBooter.java:407) >> > > >> > >> > > >> > >> > > >> > 2018-01-06 12:53:53,398 INFO [main] hbase.ResourceChecker(172): >> > > after: >> > > >> > regionserver.TestDefaultCompactSelection#testStuckStoreCompa >> ction >> > > >> > Thread=11 (was 7) >> > > >> > Potentially hanging thread: Monitor thread for TaskMonitor >> > > >> > java.lang.Thread.sleep(Native Method) >> > > >> > org.apache.hadoop.hbase.monitoring.TaskMonitor$ >> > > >> > MonitorRunnable.run(TaskMonitor.java:302) >> > > >> > java.lang.Thread.run(Thread.java:748) >> > > >> > >> > > >> > Potentially hanging thread: org.apache.hadoop.fs. >> > > FileSystem$Statistics$ >> > > >> > StatisticsDataReferenceCleaner >> > > >> > java.lang.Object.wait(Native Method) >> > > >> > java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: >> > 143) >> > > >> > java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: >> > 164) >> > > >> > org.apache.hadoop.fs.FileSystem$Statistics$ >> > > >> > StatisticsDataReferenceCleaner.run(FileSystem.java:3063) >> > > >> > java.lang.Thread.run(Thread.java:748) >> > > >> > >> > > >> > Potentially hanging thread: LruBlockCacheStatsExecutor >> > > >> > sun.misc.Unsafe.park(Native Method) >> > > >> > java.util.concurrent.locks.LockSupport.parkNanos( >> > > >> > LockSupport.java:215) >> > > >> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ >> > > >> > ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) >> > > >> > java.util.concurrent.ScheduledThreadPoolExecutor$ >> > > >> > DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) >> > > >> > java.util.concurrent.ScheduledThreadPoolExecutor$ >> > > >> > DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) >> > > >> > java.util.concurrent.ThreadPoolExecutor.getTask( >> > > >> > ThreadPoolExecutor.java:1074) >> > > >> > java.util.concurrent.ThreadPoolExecutor.runWorker( >> > > >> > ThreadPoolExecutor.java:1134) >> > > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run( >> > > >> > ThreadPoolExecutor.java:624) >> > > >> > java.lang.Thread.run(Thread.java:748) >> > > >> > >> > > >> > Potentially hanging thread: StoreOpener- >> > > 22ce1d683ba4b6b9373a3c541ebab2 >> > > >> > a2-1.LruBlockCache.EvictionThread >> > > >> > java.lang.Object.wait(Native Method) >> > > >> > org.apache.hadoop.hbase.io.hf >> ile.LruBlockCache$EvictionThre >> > > >> ad.run( >> > > >> > LruBlockCache.java:894) >> > > >> > java.lang.Thread.run(Thread.java:748) >> > > >> > - Thread LEAK? -, OpenFileDescriptor=232 (was 232), >> > > >> > MaxFileDescriptor=1048576 (was 1048576), SystemLoadAverage=204 >> (was >> > > >> 204), >> > > >> > ProcessCount=273 (was 273), AvailableMemoryMB=4049 (was 4132) >> > > >> > >> > > >> > >> > > >> > Full log attached >> > > >> > >> > > >> > Thanks, >> > > >> > >> > > >> > JMS >> > > >> > >> > > >> > >> > > >> > 2018-01-06 9:34 GMT-05:00 Mike Drob <md...@apache.org>: >> > > >> > >> > > >> >> I can reproduce the issue locally. I think it has to do with the >> > java >> > > >> >> version being used - IIRC this is related to the version of java >> > > used, >> > > >> but >> > > >> >> we can discuss in more detail on the JIRA. >> > > >> >> >> > > >> >> https://issues.apache.org/jira/browse/HBASE-19721 >> > > >> >> >> > > >> >> Thanks, JMS! >> > > >> >> >> > > >> >> On Sat, Jan 6, 2018 at 6:42 AM, Jean-Marc Spaggiari < >> > > >> >> jean-m...@spaggiari.org >> > > >> >> > wrote: >> > > >> >> >> > > >> >> > How you guys are able to get the tests running? >> > > >> >> > >> > > >> >> > For me it keeps failing on TestReversedScannerCallable. >> > > >> >> > >> > > >> >> > I tried many times, always fails in the same place. I'm >> running >> > on >> > > a >> > > >> 4GB >> > > >> >> > tmpfs. Details are below. Am I doing something wrong? >> > > >> >> > >> > > >> >> > JM >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > ./dev-support/hbasetests.sh runAllTests >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > [INFO] Running org.apache.hadoop.hbase.client.TestOperation >> > > >> >> > [INFO] >> > > >> >> > [INFO] Results: >> > > >> >> > [INFO] >> > > >> >> > [ERROR] Errors: >> > > >> >> > [ERROR] TestReversedScannerCallable.unnecessary Mockito >> > > stubbings >> > > >> » >> > > >> >> > UnnecessaryStubbing >> > > >> >> > [INFO] >> > > >> >> > [ERROR] Tests run: 245, Failures: 0, Errors: 1, Skipped: 8 >> > > >> >> > [INFO] >> > > >> >> > [INFO] >> > > >> >> > ------------------------------------------------------------ >> > > >> >> ------------ >> > > >> >> > [INFO] Reactor Summary: >> > > >> >> > [INFO] >> > > >> >> > [INFO] Apache HBase ....................................... >> > > SUCCESS >> > > >> [ >> > > >> >> > 1.409 s] >> > > >> >> > [INFO] Apache HBase - Checkstyle .......................... >> > SUCCESS >> > > [ >> > > >> >> > 1.295 s] >> > > >> >> > [INFO] Apache HBase - Build Support ....................... >> > SUCCESS >> > > [ >> > > >> >> > 0.038 s] >> > > >> >> > [INFO] Apache HBase - Error Prone Rules ................... >> > SUCCESS >> > > [ >> > > >> >> > 1.069 s] >> > > >> >> > [INFO] Apache HBase - Annotations ......................... >> > SUCCESS >> > > [ >> > > >> >> > 1.450 s] >> > > >> >> > [INFO] Apache HBase - Build Configuration ................. >> > SUCCESS >> > > [ >> > > >> >> > 0.073 s] >> > > >> >> > [INFO] Apache HBase - Shaded Protocol ..................... >> > SUCCESS >> > > [ >> > > >> >> > 14.292 s] >> > > >> >> > [INFO] Apache HBase - Common .............................. >> > SUCCESS >> > > >> >> [01:51 >> > > >> >> > min] >> > > >> >> > [INFO] Apache HBase - Metrics API ......................... >> > SUCCESS >> > > [ >> > > >> >> > 2.878 s] >> > > >> >> > [INFO] Apache HBase - Hadoop Compatibility ................ >> > SUCCESS >> > > [ >> > > >> >> > 12.216 s] >> > > >> >> > [INFO] Apache HBase - Metrics Implementation .............. >> > SUCCESS >> > > [ >> > > >> >> > 7.206 s] >> > > >> >> > [INFO] Apache HBase - Hadoop Two Compatibility ............ >> > SUCCESS >> > > [ >> > > >> >> > 12.440 s] >> > > >> >> > [INFO] Apache HBase - Protocol ............................ >> > SUCCESS >> > > [ >> > > >> >> > 0.074 s] >> > > >> >> > [INFO] Apache HBase - Client .............................. >> > FAILURE >> > > >> >> [02:10 >> > > >> >> > min] >> > > >> >> > [INFO] Apache HBase - Zookeeper ........................... >> > SKIPPED >> > > >> >> > [INFO] Apache HBase - Replication ......................... >> > SKIPPED >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > ------------------------------------------------------------ >> > > >> >> > ------------------- >> > > >> >> > Test set: org.apache.hadoop.hbase.client. >> > > TestReversedScannerCallable >> > > >> >> > ------------------------------------------------------------ >> > > >> >> > ------------------- >> > > >> >> > Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time >> elapsed: >> > > >> 1.515 s >> > > >> >> <<< >> > > >> >> > FAILURE! - in org.apache.hadoop.hbase.client >> > > >> >> .TestReversedScannerCallable >> > > >> >> > unnecessary Mockito >> > > >> >> > stubbings(org.apache.hadoop.hbase.client.TestReversedScanner >> > > >> Callable) >> > > >> >> > Time >> > > >> >> > elapsed: 0.014 s <<< ERROR! >> > > >> >> > org.mockito.exceptions.misusing.UnnecessaryStubbingException: >> > > >> >> > >> > > >> >> > Unnecessary stubbings detected in test class: >> > > >> >> TestReversedScannerCallable >> > > >> >> > Clean & maintainable test code requires zero unnecessary code. >> > > >> >> > Following stubbings are unnecessary (click to navigate to >> > relevant >> > > >> line >> > > >> >> of >> > > >> >> > code): >> > > >> >> > 1. -> at >> > > >> >> > org.apache.hadoop.hbase.client.TestReversedScannerCallable. >> > setUp( >> > > >> >> > TestReversedScannerCallable.java:66) >> > > >> >> > 2. -> at >> > > >> >> > org.apache.hadoop.hbase.client.TestReversedScannerCallable. >> > setUp( >> > > >> >> > TestReversedScannerCallable.java:68) >> > > >> >> > Please remove unnecessary stubbings. More info: javadoc for >> > > >> >> > UnnecessaryStubbingException class. >> > > >> >> > >> > > >> >> > >> > > >> >> > 2018-01-06 0:44 GMT-05:00 stack <saint....@gmail.com>: >> > > >> >> > >> > > >> >> > > On Jan 5, 2018 4:44 PM, "Apekshit Sharma" < >> a...@cloudera.com> >> > > >> wrote: >> > > >> >> > > >> > > >> >> > > bq. Care needs to be exercised backporting. Bug fixes only >> > > please. >> > > >> If >> > > >> >> in >> > > >> >> > > doubt, ping me, the RM, please. Thanks. >> > > >> >> > > In that case, shouldn't we branch out branch-2.0? We can >> then >> > do >> > > >> >> normal >> > > >> >> > > backports to branch-2 and only bug fixes to branch-2.0. >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > Don't you think we have enough branches already mighty Appy? >> > > >> >> > > >> > > >> >> > > No new features on branch-2? New features are in >> master/3.0.0 >> > > only? >> > > >> >> > > >> > > >> >> > > S >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > On Fri, Jan 5, 2018 at 9:48 AM, Andrew Purtell < >> > > >> apurt...@apache.org> >> > > >> >> > > wrote: >> > > >> >> > > >> > > >> >> > > > TestMemstoreLABWithoutPool is a flake, not a consistent >> fail. >> > > >> >> > > > >> > > >> >> > > > >> > > >> >> > > > On Fri, Jan 5, 2018 at 7:18 AM, Stack <st...@duboce.net> >> > > wrote: >> > > >> >> > > > >> > > >> >> > > > > On Thu, Jan 4, 2018 at 2:24 PM, Andrew Purtell < >> > > >> >> apurt...@apache.org> >> > > >> >> > > > > wrote: >> > > >> >> > > > > >> > > >> >> > > > > > This one is probably my fault: >> > > >> >> > > > > > >> > > >> >> > > > > > TestDefaultCompactSelection >> > > >> >> > > > > > >> > > >> >> > > > > > HBASE-19406 >> > > >> >> > > > > > >> > > >> >> > > > > > >> > > >> >> > > > > Balazs fixed it above, HBASE-19666 >> > > >> >> > > > > >> > > >> >> > > > > >> > > >> >> > > > > >> > > >> >> > > > > > It can easily be reverted. The failure of interest >> > > >> >> > > > > > is TestMemstoreLABWithoutPool.tes >> > > >> tLABChunkQueueWithMultipleMSLA >> > > >> >> Bs. >> > > >> >> > > > > > >> > > >> >> > > > > > >> > > >> >> > > > > This seems fine. Passes in nightly >> > > >> >> > > > > https://builds.apache.org/view >> /H-L/view/HBase/job/HBase% >> > > >> >> > > > > 20Nightly/job/branch-2/171/tes >> tReport/org.apache.hadoop. >> > > >> >> > > > > hbase.regionserver/TestMemstoreLABWithoutPool/ >> > > >> >> > > > > and locally against the tag. It fails consistently for >> you >> > > >> Andrew? >> > > >> >> > > > > >> > > >> >> > > > > >> > > >> >> > > > > > > Should all unit tests pass on a beta? I think so, at >> > > least >> > > >> if >> > > >> >> the >> > > >> >> > > > > > failures >> > > >> >> > > > > > > are 100% repeatable. >> > > >> >> > > > > > > >> > > >> >> > > > > > >> > > >> >> > > > > >> > > >> >> > > > > This is fair. Let me squash this RC and roll another. >> > > >> >> > > > > >> > > >> >> > > > > Will put it up in a few hours. >> > > >> >> > > > > >> > > >> >> > > > > Thanks, >> > > >> >> > > > > S >> > > >> >> > > > > >> > > >> >> > > > > >> > > >> >> > > > > >> > > >> >> > > > > > > -0 >> > > >> >> > > > > > > >> > > >> >> > > > > > > Checked sums and signatures: ok >> > > >> >> > > > > > > RAT check: ok >> > > >> >> > > > > > > Built from source: ok (8u144) >> > > >> >> > > > > > > Ran unit tests: some failures (8u144) >> > > >> >> > > > > > > >> > > >> >> > > > > > > [ERROR] TestDefaultCompactSelection.t >> > > >> >> estCompactionRatio:74-> >> > > >> >> > > > > > > TestCompactionPolicy.compactEq >> > > >> uals:182->TestCompactionPolicy. >> > > >> >> > > > > > compactEquals:201 >> > > >> >> > > > > > > expected:<[[4, 2, 1]]> but was:<[[]]> >> > > >> >> > > > > > > >> > > >> >> > > > > > > [ERROR] TestDefaultCompactSelection. >> > > >> >> > > testStuckStoreCompaction:145-> >> > > >> >> > > > > > > TestCompactionPolicy.compactEq >> > > >> uals:182->TestCompactionPolicy. >> > > >> >> > > > > > compactEquals:201 >> > > >> >> > > > > > > expected:<[[]30, 30, 30]> but was:<[[99, 30, ]30, >> 30, >> > > 30]> >> > > >> >> > > > > > > >> > > >> >> > > > > > > [ERROR] TestMemstoreLABWithoutPool. >> > > >> >> > > testLABChunkQueueWithMultipleM >> > > >> >> > > > > > SLABs:143 >> > > >> >> > > > > > > All the chunks must have been cleared >> > > >> >> > > > > > > >> > > >> >> > > > > > > >> > > >> >> > > > > > > >> > > >> >> > > > > > > On Fri, Dec 29, 2017 at 10:15 AM, Stack < >> > > st...@duboce.net> >> > > >> >> > wrote: >> > > >> >> > > > > > > >> > > >> >> > > > > > >> The first release candidate for HBase 2.0.0-beta-1 >> is >> > up >> > > >> at: >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> https://dist.apache.org/repos/ >> > > >> dist/dev/hbase/hbase-2.0.0- >> > > >> >> > > > beta-1-RC0/ >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> Maven artifacts are available from a staging >> directory >> > > >> here: >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> https://repository.apache.org/ >> content/repositories/ >> > > >> >> > > > > orgapachehbase-1188 >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> All was signed with my key at 8ACC93D2 [1] >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> I tagged the RC as 2.0.0-beta-1-RC0 >> > > >> >> > > > > > >> (0907563eb72697b394b8b960fe54887d6ff304fd) >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> hbase-2.0.0-beta-1 is our first beta release. It >> > > includes >> > > >> all >> > > >> >> > that >> > > >> >> > > > was >> > > >> >> > > > > > in >> > > >> >> > > > > > >> previous alphas (new assignment manager, offheap >> > > >> read/write >> > > >> >> > path, >> > > >> >> > > > > > >> in-memory >> > > >> >> > > > > > >> compactions, etc.). The APIs and feature-set are >> > sealed. >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> hbase-2.0.0-beta-1 is a not-for-production preview >> of >> > > >> >> > hbase-2.0.0. >> > > >> >> > > > It >> > > >> >> > > > > is >> > > >> >> > > > > > >> meant for devs and downstreamers to test drive and >> > flag >> > > us >> > > >> >> if we >> > > >> >> > > > > messed >> > > >> >> > > > > > up >> > > >> >> > > > > > >> on anything ahead of our rolling GAs. We are >> > particular >> > > >> >> > interested >> > > >> >> > > > in >> > > >> >> > > > > > >> hearing from Coprocessor developers. >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> The list of features addressed in 2.0.0 so far can >> be >> > > >> found >> > > >> >> here >> > > >> >> > > > [3]. >> > > >> >> > > > > > >> There >> > > >> >> > > > > > >> are thousands. The list of ~2k+ fixes in 2.0.0 >> > > exclusively >> > > >> >> can >> > > >> >> > be >> > > >> >> > > > > found >> > > >> >> > > > > > >> here [4] (My JIRA JQL foo is a bit dodgy -- >> forgive me >> > > if >> > > >> >> > > mistakes). >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> I've updated our overview doc. on the state of >> 2.0.0 >> > > [6]. >> > > >> >> We'll >> > > >> >> > do >> > > >> >> > > > one >> > > >> >> > > > > > >> more >> > > >> >> > > > > > >> beta before we put up our first 2.0.0 Release >> > Candidate >> > > by >> > > >> >> the >> > > >> >> > end >> > > >> >> > > > of >> > > >> >> > > > > > >> January, 2.0.0-beta-2. Its focus will be making it >> so >> > > >> users >> > > >> >> can >> > > >> >> > do >> > > >> >> > > a >> > > >> >> > > > > > >> rolling upgrade on to hbase-2.x from hbase-1.x (and >> > any >> > > >> bug >> > > >> >> > fixes >> > > >> >> > > > > found >> > > >> >> > > > > > >> running beta-1). Here is the list of what we have >> > > >> targeted so >> > > >> >> > far >> > > >> >> > > > for >> > > >> >> > > > > > >> beta-2 [5]. Check it out. >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> One knownissue is that the User API has not been >> > > properly >> > > >> >> > filtered >> > > >> >> > > > so >> > > >> >> > > > > it >> > > >> >> > > > > > >> shows more than just InterfaceAudience Public >> content >> > > >> >> > > (HBASE-19663, >> > > >> >> > > > to >> > > >> >> > > > > > be >> > > >> >> > > > > > >> fixed by beta-2). >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> Please take this beta for a spin. Please vote on >> > whether >> > > >> it >> > > >> >> ok >> > > >> >> > to >> > > >> >> > > > put >> > > >> >> > > > > > out >> > > >> >> > > > > > >> this RC as our first beta (Note CHANGES has not yet >> > been >> > > >> >> > updated). >> > > >> >> > > > Let >> > > >> >> > > > > > the >> > > >> >> > > > > > >> VOTE be open for 72 hours (Monday) >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> Thanks, >> > > >> >> > > > > > >> Your 2.0.0 Release Manager >> > > >> >> > > > > > >> >> > > >> >> > > > > > >> 1. http://pgp.mit.edu/pks/lookup?op=get&search= >> > > >> >> > 0x9816C7FC8ACC93D2 >> > > >> >> > > > > > >> 3. https://goo.gl/scYjJr >> > > >> >> > > > > > >> 4. https://goo.gl/dFFT8b >> > > >> >> > > > > > >> 5. https://issues.apache.org/jira >> > > >> /projects/HBASE/versions/ >> > > >> >> > > 12340862 >> > > >> >> > > > > > >> 6. https://docs.google.com/document/d/ >> > > >> >> > > 1WCsVlnHjJeKUcl7wHwqb4z9iEu_ >> > > >> >> > > > > > >> ktczrlKHK8N4SZzs/ >> > > >> >> > > > > > >> <https://docs.google.com/document/d/ >> > > >> >> > 1WCsVlnHjJeKUcl7wHwqb4z9iEu_ >> > > >> >> > > > > > ktczrlKHK8N4SZzs/> >> > > >> >> > > > > > >> >> > > >> >> > > > > > > >> > > >> >> > > > > > > >> > > >> >> > > > > > > >> > > >> >> > > > > > > -- >> > > >> >> > > > > > > Best regards, >> > > >> >> > > > > > > Andrew >> > > >> >> > > > > > > >> > > >> >> > > > > > > Words like orphans lost among the crosstalk, meaning >> > torn >> > > >> from >> > > >> >> > > > truth's >> > > >> >> > > > > > > decrepit hands >> > > >> >> > > > > > > - A23, Crosstalk >> > > >> >> > > > > > > >> > > >> >> > > > > > >> > > >> >> > > > > > >> > > >> >> > > > > > >> > > >> >> > > > > > -- >> > > >> >> > > > > > Best regards, >> > > >> >> > > > > > Andrew >> > > >> >> > > > > > >> > > >> >> > > > > > Words like orphans lost among the crosstalk, meaning >> torn >> > > >> from >> > > >> >> > > truth's >> > > >> >> > > > > > decrepit hands >> > > >> >> > > > > > - A23, Crosstalk >> > > >> >> > > > > > >> > > >> >> > > > > >> > > >> >> > > > >> > > >> >> > > > >> > > >> >> > > > >> > > >> >> > > > -- >> > > >> >> > > > Best regards, >> > > >> >> > > > Andrew >> > > >> >> > > > >> > > >> >> > > > Words like orphans lost among the crosstalk, meaning torn >> > from >> > > >> >> truth's >> > > >> >> > > > decrepit hands >> > > >> >> > > > - A23, Crosstalk >> > > >> >> > > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > -- >> > > >> >> > > >> > > >> >> > > -- Appy >> > > >> >> > > >> > > >> >> > >> > > >> >> >> > > >> > >> > > >> > >> > > >> >> > > > >> > > > >> > > >> > >> > >> > >> > -- >> > >> > -- Appy >> > >> > >
