[ https://issues.apache.org/jira/browse/HBASE-18693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16199628#comment-16199628 ]
Hadoop QA commented on HBASE-18693: ----------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 37s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 19s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 48s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 55s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m 10s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m 6s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m 9s{color} | {color:red} The patch generated 3 new + 334 unchanged - 1 fixed = 337 total (was 335) {color} | | {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red} 0m 5s{color} | {color:red} The patch generated 1 new + 743 unchanged - 0 fixed = 744 total (was 743) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m 46s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 60m 34s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 2m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 11m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 15s{color} | {color:green} hbase-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}124m 16s{color} | {color:red} hbase-server in the patch failed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 43s{color} | {color:red} hbase-shell in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m 18s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}244m 9s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.coprocessor.TestRegionObserverForAddingMutationsFromCoprocessors | | | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 | | | hadoop.hbase.client.TestShell | | Timed out junit tests | org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay | | | org.apache.hadoop.hbase.regionserver.wal.TestAsyncWALReplay | | | org.apache.hadoop.hbase.regionserver.TestSplitLogWorker | | | org.apache.hadoop.hbase.regionserver.TestRowTooBig | | | org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence | | | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy | | | org.apache.hadoop.hbase.client.TestAsyncTableScanAll | | | org.apache.hadoop.hbase.client.TestMultipleTimestamps | | | org.apache.hadoop.hbase.client.TestFromClientSide | | | org.apache.hadoop.hbase.regionserver.wal.TestFSHLog | | | org.apache.hadoop.hbase.trace.TestHTraceHooks | | | org.apache.hadoop.hbase.regionserver.TestRemoveRegionMetrics | | | org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics | | | org.apache.hadoop.hbase.namespace.TestNamespaceAuditor | | | org.apache.hadoop.hbase.TestHBaseTestingUtility | | | org.apache.hadoop.hbase.client.TestAsyncTableBatch | | | org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi | | | org.apache.hadoop.hbase.client.TestAsyncProcedureAdminApi | | | org.apache.hadoop.hbase.regionserver.TestTags | | | org.apache.hadoop.hbase.regionserver.TestTimestampFilterSeekHint | | | org.apache.hadoop.hbase.client.TestAsyncSnapshotAdminApi | | | org.apache.hadoop.hbase.client.TestAsyncNonMetaRegionLocator | | | org.apache.hadoop.hbase.regionserver.wal.TestWALReplay | | | org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi | | | org.apache.hadoop.hbase.regionserver.TestScannerHeartbeatMessages | | | org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi | | | org.apache.hadoop.hbase.client.TestAsyncTableScanRenewLease | | | org.apache.hadoop.hbase.client.TestAsyncTableAdminApi | | | org.apache.hadoop.hbase.client.TestAsyncRegionLocatorTimeout | | | org.apache.hadoop.hbase.wal.TestWALSplitCompressed | | | org.apache.hadoop.hbase.filter.TestFilterWithScanLimits | | | org.apache.hadoop.hbase.client.TestScannersFromClientSide2 | | | org.apache.hadoop.hbase.regionserver.TestRegionReplicas | | | org.apache.hadoop.hbase.client.TestMultiRespectsLimits | | | org.apache.hadoop.hbase.regionserver.TestMutateRowsRecovery | | | org.apache.hadoop.hbase.client.TestServerBusyException | | | org.apache.hadoop.hbase.regionserver.TestRegionServerAbort | | | org.apache.hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput | | | org.apache.hadoop.hbase.regionserver.TestCompactSplitThread | | | org.apache.hadoop.hbase.regionserver.wal.TestLogRollAbort | | | org.apache.hadoop.hbase.regionserver.TestRegionReplicasWithModifyTable | | | org.apache.hadoop.hbase.client.TestFromClientSide3 | | | org.apache.hadoop.hbase.regionserver.TestWalAndCompactingMemStoreFlush | | | org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput | | | org.apache.hadoop.hbase.client.TestScannerTimeout | | | org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster | | | org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss | | | org.apache.hadoop.hbase.regionserver.TestCompactionArchiveConcurrentClose | | | org.apache.hadoop.hbase.io.encoding.TestDataBlockEncoders | | | org.apache.hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | | org.apache.hadoop.hbase.client.TestSplitOrMergeStatus | | | org.apache.hadoop.hbase.regionserver.TestRegionServerReportForDuty | | | org.apache.hadoop.hbase.regionserver.wal.TestLogRolling | | | org.apache.hadoop.hbase.client.TestAsyncSingleRequestRpcRetryingCaller | | | org.apache.hadoop.hbase.regionserver.throttle.TestCompactionWithThroughputController | | | org.apache.hadoop.hbase.client.TestAdmin2 | | | org.apache.hadoop.hbase.client.TestIncrementFromClientSideWithCoprocessor | | | org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush | | | org.apache.hadoop.hbase.io.hfile.TestScannerSelectionUsingTTL | | | org.apache.hadoop.hbase.client.TestAdmin1 | | | org.apache.hadoop.hbase.regionserver.TestScannerWithBulkload | | | org.apache.hadoop.hbase.regionserver.wal.TestLogRollingNoCluster | | | org.apache.hadoop.hbase.regionserver.TestJoinedScanners | | | org.apache.hadoop.hbase.client.TestAsyncBufferMutator | | | org.apache.hadoop.hbase.regionserver.throttle.TestFlushWithThroughputController | | | org.apache.hadoop.hbase.regionserver.TestDeleteMobTable | | | org.apache.hadoop.hbase.filter.TestFilterListOrOperatorWithBlkCnt | | | org.apache.hadoop.hbase.TestAcidGuarantees | | | org.apache.hadoop.hbase.client.TestSnapshotMetadata | | | org.apache.hadoop.hbase.io.encoding.TestChangingEncoding | | | org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks | | | org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor | | | org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster | | | org.apache.hadoop.hbase.regionserver.TestCompoundBloomFilter | | | org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoadWithOldClient | | | org.apache.hadoop.hbase.client.TestEnableTable | | | org.apache.hadoop.hbase.regionserver.TestRecoveredEdits | | | org.apache.hadoop.hbase.client.TestAsyncTable | | | org.apache.hadoop.hbase.filter.TestMultiRowRangeFilter | | | org.apache.hadoop.hbase.client.TestHTableMultiplexerFlushCache | | | org.apache.hadoop.hbase.client.TestClientOperationInterrupt | | | org.apache.hadoop.hbase.filter.TestColumnRangeFilter | | | org.apache.hadoop.hbase.client.TestLeaseRenewal | | | org.apache.hadoop.hbase.regionserver.TestNewVersionBehaviorFromClientSide | | | org.apache.hadoop.hbase.client.TestTimestampsFilter | | | org.apache.hadoop.hbase.regionserver.TestRSKilledWhenInitializing | | | org.apache.hadoop.hbase.client.TestHCM | | | org.apache.hadoop.hbase.io.encoding.TestLoadAndSwitchEncodeOnDisk | | | org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite | | | org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded | | | org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner | | | org.apache.hadoop.hbase.client.TestAsyncResultScannerCursor | | | org.apache.hadoop.hbase.client.TestRawAsyncTableScan | | | org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClient | | | org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientWithRegionReplicas | | | org.apache.hadoop.hbase.client.TestMvccConsistentScanner | | | org.apache.hadoop.hbase.regionserver.TestClusterId | | | org.apache.hadoop.hbase.regionserver.TestRegionServerHostname | | | org.apache.hadoop.hbase.regionserver.TestMobStoreScanner | | | org.apache.hadoop.hbase.client.TestSmallReversedScanner | | | org.apache.hadoop.hbase.client.TestTableFavoredNodes | | | org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient | | | org.apache.hadoop.hbase.client.TestTableSnapshotScanner | | | org.apache.hadoop.hbase.client.TestAsyncTableScan | | | org.apache.hadoop.hbase.client.TestCloneSnapshotFromClientWithRegionReplicas | | | org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient | | | org.apache.hadoop.hbase.regionserver.TestSettingTimeoutOnBlockingPoint | | | org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction | | | org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRollPeriod | | | org.apache.hadoop.hbase.TestMetaTableAccessor | | | org.apache.hadoop.hbase.client.TestResultScannerCursor | | | org.apache.hadoop.hbase.regionserver.wal.TestAsyncWALReplayCompressed | | | org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation | | | org.apache.hadoop.hbase.regionserver.TestCompactionFileNotFound | | | org.apache.hadoop.hbase.io.hfile.TestHFileBlock | | | org.apache.hadoop.hbase.client.TestRawAsyncScanCursor | | | org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient | | | org.apache.hadoop.hbase.client.TestAsyncTableScanner | | | org.apache.hadoop.hbase.regionserver.TestHStore | | | org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover | | | org.apache.hadoop.hbase.client.TestIllegalTableDescriptor | | | org.apache.hadoop.hbase.wal.TestBoundedRegionGroupingStrategy | | | org.apache.hadoop.hbase.client.TestMobSnapshotFromClient | | | org.apache.hadoop.hbase.client.TestIncrementsFromClientSide | | | org.apache.hadoop.hbase.client.TestHTableMultiplexer | | | org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd | | | org.apache.hadoop.hbase.client.TestSnapshotFromClientWithRegionReplicas | | | org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad | | | org.apache.hadoop.hbase.regionserver.wal.TestLogRollPeriod | | | org.apache.hadoop.hbase.regionserver.TestCompactionState | | | org.apache.hadoop.hbase.client.TestAsyncAdminBuilder | | | org.apache.hadoop.hbase.client.TestMetaCache | | | org.apache.hadoop.hbase.client.TestScannersFromClientSide | | | org.apache.hadoop.hbase.regionserver.wal.TestWALReplayCompressed | | | org.apache.hadoop.hbase.regionserver.wal.TestSecureAsyncWALReplay | | | org.apache.hadoop.hbase.regionserver.TestMobStoreCompaction | | | org.apache.hadoop.hbase.client.TestSnapshotFromClient | | | org.apache.hadoop.hbase.regionserver.TestAtomicOperation | | | org.apache.hadoop.hbase.regionserver.TestMajorCompaction | | | org.apache.hadoop.hbase.io.encoding.TestEncodedSeekers | | | org.apache.hadoop.hbase.client.TestAsyncClusterAdminApi2 | | | org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 | | JIRA Issue | HBASE-18693 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12891340/HBASE-18693.master.002.patch | | Optional Tests | asflicense shadedjars javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc rubocop ruby_lint | | uname | Linux d0c947d998f5 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / 690ff38 | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC3 | | rubocop | v0.50.0 | | rubocop | https://builds.apache.org/job/PreCommit-HBASE-Build/9030/artifact/patchprocess/diff-patch-rubocop.txt | | ruby-lint | v2.3.1 | | ruby-lint | https://builds.apache.org/job/PreCommit-HBASE-Build/9030/artifact/patchprocess/diff-patch-ruby-lint.txt | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/9030/artifact/patchprocess/patch-unit-hbase-server.txt | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/9030/artifact/patchprocess/patch-unit-hbase-shell.txt | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/9030/testReport/ | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/9030/console | | Powered by | Apache Yetus 0.4.0 http://yetus.apache.org | This message was automatically generated. > adding an option to restore_snapshot to move mob files from archive dir to > working dir > -------------------------------------------------------------------------------------- > > Key: HBASE-18693 > URL: https://issues.apache.org/jira/browse/HBASE-18693 > Project: HBase > Issue Type: Improvement > Components: mob > Affects Versions: 2.0.0-alpha-2 > Reporter: huaxiang sun > Assignee: huaxiang sun > Attachments: HBASE-18693.master.001.patch, > HBASE-18693.master.002.patch > > > Today, there is a single mob region where mob files for all user regions are > saved. There could be many files (one million) in a single mob directory. > When one mob table is restored or cloned from snapshot, links are created for > these mob files. This creates a scaling issue for mob compaction. In mob > compaction's select() logic, for each hFileLink, it needs to call NN's > getFileStatus() to get the size of the linked hfile. Assume that one such > call takes 20ms, 20ms * 1000000 = 6 hours. > To avoid this overhead, we want to add an option so that restore_snapshot can > move mob files from archive dir to working dir. clone_snapshot is more > complicated as it can clone a snapshot to a different table so moving that > can destroy the snapshot. No option will be added for clone_snapshot. -- This message was sent by Atlassian JIRA (v6.4.14#64029)