[GitHub] [hbase] syedmurtazahassan commented on issue #386: HBASE-22610 [BucketCache] Rename hbase.offheapcache.minblocksize.
syedmurtazahassan commented on issue #386: HBASE-22610 [BucketCache] Rename hbase.offheapcache.minblocksize. URL: https://github.com/apache/hbase/pull/386#issuecomment-511689886 @Reidd Thanks for the detailed reply. I will look into this and also see if there is an already existing example and whow it is preferred. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sunhelly commented on a change in pull request #375: HBASE-21426 TestEncryptionKeyRotation.testCFKeyRotation is flaky
sunhelly commented on a change in pull request #375: HBASE-21426
TestEncryptionKeyRotation.testCFKeyRotation is flaky
URL: https://github.com/apache/hbase/pull/375#discussion_r303745811
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEncryptionKeyRotation.java
##
@@ -91,6 +92,7 @@ public static void setUp() throws Exception {
conf.setInt("hfile.format.version", 3);
conf.set(HConstants.CRYPTO_KEYPROVIDER_CONF_KEY,
KeyProviderForTesting.class.getName());
conf.set(HConstants.CRYPTO_MASTERKEY_NAME_CONF_KEY, "hbase");
+conf.setInt("hbase.hfile.compaction.discharger.interval", 10 * 60 * 1000);
Review comment:
@infraio Thanks for reviewing.
> Why this config change?
I changed this to let the compacted files be cleaned up slower than
normal(10min instead of 2min). Because when I run testCFKeyRotation() for 100
times, I have met assertion fail at `List compactedPaths =
findCompactedStorefilePaths(htd.getTableName());
assertTrue(compactedPaths.size() > 0);`. So this test case may also be
flaky at this assertion.
Enlarging the compaction discharger interval can ensure compacted files
still be there after major compaction is completed but compacted paths are not
checked in testCFKeyRotation().
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22661) list_regions command in hbase shell is broken
[
https://issues.apache.org/jira/browse/HBASE-22661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885872#comment-16885872
]
Toshihiro Suzuki commented on HBASE-22661:
--
+1 Thank you for the patch.
> list_regions command in hbase shell is broken
> -
>
> Key: HBASE-22661
> URL: https://issues.apache.org/jira/browse/HBASE-22661
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Toshihiro Suzuki
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-22661.patch
>
>
> I faced the following error in the master branch:
> {code}
> hbase(main):001:0> create "test", "cf"
> 2019-07-07 23:24:15,254 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> Created table test
> Took 6.5678 seconds
> => Hbase::Table - test
> hbase(main):002:0> list_regions "test"
> ERROR: undefined method `getClusterStatus' for
> #
> Did you mean? get_cluster_metrics
> For usage try 'help "list_regions"'
> Took 0.1997 seconds
> {code}
> I didn't check if the other branches have the same issue.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (HBASE-22697) when RegionServerStoppedException is received, the client should clear meta cache
Junhong Xu created HBASE-22697: -- Summary: when RegionServerStoppedException is received, the client should clear meta cache Key: HBASE-22697 URL: https://issues.apache.org/jira/browse/HBASE-22697 Project: HBase Issue Type: Improvement Reporter: Junhong Xu Assignee: Junhong Xu but now it will retry utill exhausted -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22648) Snapshot TTL
[ https://issues.apache.org/jira/browse/HBASE-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-22648: - Attachment: HBASE-22648-master-v5.patch > Snapshot TTL > > > Key: HBASE-22648 > URL: https://issues.apache.org/jira/browse/HBASE-22648 > Project: HBase > Issue Type: New Feature > Components: snapshots >Affects Versions: 3.0.0 >Reporter: Andrew Purtell >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-22648-master-v2.patch, > HBASE-22648-master-v3.patch, HBASE-22648-master-v4.patch, > HBASE-22648-master-v5.patch, HBASE-22648-master.patch, Screen Shot 2019-07-10 > at 8.49.13 PM.png, Screen Shot 2019-07-10 at 8.52.30 PM.png, Screen Shot > 2019-07-10 at 9.06.36 PM.png, Screen Shot 2019-07-16 at 11.06.03 AM.png > > > Snapshots have a lifecycle that is independent from the table from which they > are created. Although data in a table may be stored with TTL the data files > containing them become frozen by the snapshot. Space consumed by expired > cells will not be reclaimed by normal table housekeeping like compaction. > While this is expected it can be inconvenient at scale. When many snapshots > are under management and the data in various tables is expired by TTL some > notion of optional TTL (and optional default TTL) for snapshots could be > useful. It will help prevent the accumulation of junk files by automatically > dropping the snapshot after the assigned TTL, making their data files > eligible for cleaning. More comprehensive snapshot lifecycle management may > be considered in the future but this one case is expected to be immediately > useful given TTls on data are commonly applied for similar convenience. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[ https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22527: --- Attachment: HBASE-22527.master.003.patch > [hbck2] Add a master web ui to show the problematic regions > --- > > Key: HBASE-22527 > URL: https://issues.apache.org/jira/browse/HBASE-22527 > Project: HBase > Issue Type: Improvement > Components: hbase-operator-tools, hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: 2019-07-16 08-50-07屏幕截图.png, > HBASE-22527.master.001.patch, HBASE-22527.master.002.patch, > HBASE-22527.master.003.patch > > > On our cluster which based 2.2.0, we found one problem: there are some opened > regions which had wrong regionserver in meta. The regionserver is not exist. > We used hbck2 to fix them by the following steps. > # disable table > # bypass the stucked close region procedure (as the target regionserver is > not exist) and disable table procedure. > # setRegionState to CLOSED. > # setTableState to DISABLED. > # enable table > We found this problem by scan the hbase:meta. I thought we should add this > feature to hbck2. The we can use hbck2 to find this problem. Thanks. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22648) Snapshot TTL
[ https://issues.apache.org/jira/browse/HBASE-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-22648: - Attachment: Screen Shot 2019-07-16 at 11.06.03 AM.png > Snapshot TTL > > > Key: HBASE-22648 > URL: https://issues.apache.org/jira/browse/HBASE-22648 > Project: HBase > Issue Type: New Feature > Components: snapshots >Affects Versions: 3.0.0 >Reporter: Andrew Purtell >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-22648-master-v2.patch, > HBASE-22648-master-v3.patch, HBASE-22648-master-v4.patch, > HBASE-22648-master.patch, Screen Shot 2019-07-10 at 8.49.13 PM.png, Screen > Shot 2019-07-10 at 8.52.30 PM.png, Screen Shot 2019-07-10 at 9.06.36 PM.png, > Screen Shot 2019-07-16 at 11.06.03 AM.png > > > Snapshots have a lifecycle that is independent from the table from which they > are created. Although data in a table may be stored with TTL the data files > containing them become frozen by the snapshot. Space consumed by expired > cells will not be reclaimed by normal table housekeeping like compaction. > While this is expected it can be inconvenient at scale. When many snapshots > are under management and the data in various tables is expired by TTL some > notion of optional TTL (and optional default TTL) for snapshots could be > useful. It will help prevent the accumulation of junk files by automatically > dropping the snapshot after the assigned TTL, making their data files > eligible for cleaning. More comprehensive snapshot lifecycle management may > be considered in the future but this one case is expected to be immediately > useful given TTls on data are commonly applied for similar convenience. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-21665) OfflineMetaRepair tool fails with NPE
[
https://issues.apache.org/jira/browse/HBASE-21665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885847#comment-16885847
]
Sean Busbey commented on HBASE-21665:
-
the current patches on HBASE-22690 doesn't just deprecate it in 2.x, it
effectively removes the tool and replaces it with a message that says where to
find the equivalent functionality in hbck2.
> OfflineMetaRepair tool fails with NPE
> -
>
> Key: HBASE-21665
> URL: https://issues.apache.org/jira/browse/HBASE-21665
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 2.1.0, 2.0.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HBASE-21665.patch
>
>
> OfflineMetaRepair fails with NPE, execute below command
> hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair -fix
>
> {noformat}
> 2019-01-02 16:22:56,387 INFO [main] regionserver.HRegion: Opened 1588230740;
> next sequenceid=28
> 2019-01-02 16:22:56,459 ERROR [main] hbck.OfflineMetaRepair: Bailed out due
> to:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.getOrMakeChunk(MemStoreLABImpl.java:335)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:193)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:115)
> at
> org.apache.hadoop.hbase.regionserver.Segment.maybeCloneWithAllocator(Segment.java:183)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.maybeCloneWithAllocator(AbstractMemStore.java:334)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.doAdd(AbstractMemStore.java:157)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.doAddOrUpsert(AbstractMemStore.java:147)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:117)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:111)
> at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:750)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyToMemStore(HRegion.java:4435)
> at org.apache.hadoop.hbase.regionserver.HRegion.access$500(HRegion.java:228)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.applyFamilyMapToMemStore(HRegion.java:3495)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.lambda$writeMiniBatchOperationsToMemStore$0(HRegion.java:3186)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.visitBatchOperations(HRegion.java:3119)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.writeMiniBatchOperationsToMemStore(HRegion.java:3178)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.writeMiniBatchOperationsToMemStore(HRegion.java:3660)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4073)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4006)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3937)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3928)
> at org.apache.hadoop.hbase.util.HBaseFsck.rebuildMeta(HBaseFsck.java:1665)
> at
> org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair.main(OfflineMetaRepair.java:121)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22688) [HBCK2] Add filesystem fixup to hbck2
[ https://issues.apache.org/jira/browse/HBASE-22688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885845#comment-16885845 ] stack commented on HBASE-22688: --- Tried to test these changes against an hbase1 big cluster but unfortunately having trouble providing hbase dependencies and compatible snappy (this cluster is all snappied...). > [HBCK2] Add filesystem fixup to hbck2 > - > > Key: HBASE-22688 > URL: https://issues.apache.org/jira/browse/HBASE-22688 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Reporter: stack >Priority: Major > Attachments: > 0001-HBASE-22688-HBCK2-Add-filesystem-fixup-to-hbck2.patch > > > Poking around, was going to add a 'filesystem' command to hbck2. It would > take nothing or a list of one or more table names and optionally a path to a > region. If nothing passed, it would review the fs for corrupt hfiles and bad > links reporting the bad as it went. If --fix is passed, it will sideline > corrupt files and mis-links. Would advise reopen/rolling-restart if change > made. For starters would make use of the hbck1 fs tools added by HBASE-22680. > Above would exploit the following facillty from hbck1. > -checkCorruptHFiles > -sidelineCorruptHFiles > -fixVersionFile > -fixReferenceFiles > -fixHFileLinks > This would address the following from parent issue at least: > * Fix hfile link problems... > * Enumerate store files to determine file level corruption... > I noticed that the 'read-only' hbck1 tool HBaseFsck which is present in the > hbase2 codebase actually allows fixes in the filesystem; i.e. you'd run > HBaseFsck and pass say -fixVersionFile. This is mixed in w/ all the other > hbck1 noise but an operator should be able to make progress with what is > there regards hfile corruption, bad links, etc. In other words, could just > doc this facility in the hbck2 tool point folks to the latent hbck1 HBaseFsck > if they need to do fixup of the fs (Would be confusing but it exists). -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-21665) OfflineMetaRepair tool fails with NPE
[
https://issues.apache.org/jira/browse/HBASE-21665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885843#comment-16885843
]
Pankaj Kumar commented on HBASE-21665:
--
Yeah, closing it as "won't fix" should be fine.
But if OMR is just going to be depricated in 2.x then we shouldn't throw NPE.
https://issues.apache.org/jira/browse/HBASE-22690?focusedCommentId=16884524&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16884524
> OfflineMetaRepair tool fails with NPE
> -
>
> Key: HBASE-21665
> URL: https://issues.apache.org/jira/browse/HBASE-21665
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 2.1.0, 2.0.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HBASE-21665.patch
>
>
> OfflineMetaRepair fails with NPE, execute below command
> hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair -fix
>
> {noformat}
> 2019-01-02 16:22:56,387 INFO [main] regionserver.HRegion: Opened 1588230740;
> next sequenceid=28
> 2019-01-02 16:22:56,459 ERROR [main] hbck.OfflineMetaRepair: Bailed out due
> to:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.getOrMakeChunk(MemStoreLABImpl.java:335)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:193)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:115)
> at
> org.apache.hadoop.hbase.regionserver.Segment.maybeCloneWithAllocator(Segment.java:183)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.maybeCloneWithAllocator(AbstractMemStore.java:334)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.doAdd(AbstractMemStore.java:157)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.doAddOrUpsert(AbstractMemStore.java:147)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:117)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:111)
> at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:750)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.applyToMemStore(HRegion.java:4435)
> at org.apache.hadoop.hbase.regionserver.HRegion.access$500(HRegion.java:228)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.applyFamilyMapToMemStore(HRegion.java:3495)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.lambda$writeMiniBatchOperationsToMemStore$0(HRegion.java:3186)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.visitBatchOperations(HRegion.java:3119)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.writeMiniBatchOperationsToMemStore(HRegion.java:3178)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.writeMiniBatchOperationsToMemStore(HRegion.java:3660)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4073)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4006)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3937)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3928)
> at org.apache.hadoop.hbase.util.HBaseFsck.rebuildMeta(HBaseFsck.java:1665)
> at
> org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair.main(OfflineMetaRepair.java:121)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22652) Flakey TestLockManager; test timed out after 780 seconds
[
https://issues.apache.org/jira/browse/HBASE-22652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885842#comment-16885842
]
stack commented on HBASE-22652:
---
Opened HBASE-22696 to deal w/ LockProcedure suggestions made above by [~Apache9]
> Flakey TestLockManager; test timed out after 780 seconds
>
>
> Key: HBASE-22652
> URL: https://issues.apache.org/jira/browse/HBASE-22652
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Affects Versions: 2.1.5
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments:
> 0001-HBASE-22652-Flakey-TestLockManager-test-timed-out-af.patch,
> HBASE-22652.branch-2.1.001.patch
>
>
> In nightly 1318 on branch-2.1, TestLockManager failed. It was only fail. It
> fails on occasion. In this instance, test failed like this:
> {code}
> Error Message
> test timed out after 780 seconds
> Stacktrace
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.locking.TestLockManager.tearDown(TestLockManager.java:105)
> {code}
> Digging in log, before the timeout in teardown, you see this:
> {code}
> 2019-07-01 22:45:57,849 INFO [PEWorker-1]
> procedure2.ProcedureExecutor$WorkerThread(2041): ASSERT pid=11
> java.lang.AssertionError
> at
> org.apache.hadoop.hbase.procedure2.TimeoutExecutorThread.add(TimeoutExecutorThread.java:81)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1765)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1462)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1200(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:2039)
> {code}
> i.e. an assert trips. The assert is doing this:
> {code}
> assert procedure.getState() == ProcedureState.WAITING_TIMEOUT;
> {code}
> The procedure is expected to be in WAITING_TIMEOUT state when we go to add it
> to the timeout executor. In the log snippet above, we see the thread is in
> RUNNABLE state. But even on successful run it is in RUNNABLE state. So what
> is going on.
> I think I figured it accidentally.
> I'd added a pause at the assert and when the assert ran, procedure was in
> RUNNABLE state and it would fail every time. So, it is a timing issue.
> I added logging of the setting of state on the procedure. There weren't many
> procedures so was easy to follow. Here is what I saw (log included emission
> of stack trace):
> {code}
> 2019-07-02 19:50:14,941 DEBUG [PEWorker-1] locking.LockProcedure(313): LOCKED
> pid=11, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure,
> namespace=namespace, type=EXCLUSIVE
> 2019-07-02 19:50:14,943 INFO [PEWorker-1] procedure2.Procedure(793): SET
> STATE ON pid=11, state=WAITING_TIMEOUT, locked=true;
> org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
> type=EXCLUSIVE
> java.lang.Exception
> at
> org.apache.hadoop.hbase.procedure2.Procedure.setState(Procedure.java:793)
> at
> org.apache.hadoop.hbase.master.locking.LockProcedure.execute(LockProcedure.java:249)
> at
> org.apache.hadoop.hbase.master.locking.LockProcedure.execute(LockProcedure.java:56)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:966)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1723)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1462)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1200(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:2039)
> 2019-07-02 19:50:14,944 INFO [Time-limited test] procedure2.Procedure(793):
> SET STATE ON pid=11, state=RUNNABLE, locked=true;
> org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
> type=EXCLUSIVE
> java.lang.Exception
> at
> org.apache.hadoop.hbase.procedure2.Procedure.setState(Procedure.java:793)
> at
> org.apache.hadoop.hbase.master.locking.LockProcedure.unlock(LockProcedure.java:225)
> at
> org.apache.hadoop.hbase.master.locking.LockManager$MasterLock.release(LockManager.java:185)
> at
> org.apache.hadoop.hbase.master.locking.TestLockManager.testMasterLockAcquire(TestLockManager.java:124)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.refl
[jira] [Updated] (HBASE-22696) LockProcedure cleanups
[ https://issues.apache.org/jira/browse/HBASE-22696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22696: -- Environment: (was: Our Duo made some comments in the parent issue: "The unlock method should not be able to execute at the same time with the execute method? Otherwise there may be logic error?" ) > LockProcedure cleanups > -- > > Key: HBASE-22696 > URL: https://issues.apache.org/jira/browse/HBASE-22696 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > > And I think we should remove the dependency on it for our internal usage. I > think the main usage is snapshot... -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22696) LockProcedure cleanups
[ https://issues.apache.org/jira/browse/HBASE-22696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22696: -- Description: Our Duo made some comments in the parent issue: * "The unlock method should not be able to execute at the same time with the execute method? Otherwise there may be logic error?" * "And I think we should remove the dependency on it for our internal usage. I think the main usage is snapshot..." was:And I think we should remove the dependency on it for our internal usage. I think the main usage is snapshot... > LockProcedure cleanups > -- > > Key: HBASE-22696 > URL: https://issues.apache.org/jira/browse/HBASE-22696 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > > Our Duo made some comments in the parent issue: > * "The unlock method should not be able to execute at the same time with the > execute method? Otherwise there may be logic error?" > * "And I think we should remove the dependency on it for our internal usage. > I think the main usage is snapshot..." -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22696) LockProcedure cleanups
[ https://issues.apache.org/jira/browse/HBASE-22696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22696: -- Description: And I think we should remove the dependency on it for our internal usage. I think the main usage is snapshot... > LockProcedure cleanups > -- > > Key: HBASE-22696 > URL: https://issues.apache.org/jira/browse/HBASE-22696 > Project: HBase > Issue Type: Sub-task > Environment: Our Duo made some comments in the parent issue: "The > unlock method should not be able to execute at the same time with the execute > method? Otherwise there may be logic error?" >Reporter: stack >Priority: Major > > And I think we should remove the dependency on it for our internal usage. I > think the main usage is snapshot... -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HBASE-22696) LockProcedure cleanups
stack created HBASE-22696: - Summary: LockProcedure cleanups Key: HBASE-22696 URL: https://issues.apache.org/jira/browse/HBASE-22696 Project: HBase Issue Type: Sub-task Environment: Our Duo made some comments in the parent issue: "The unlock method should not be able to execute at the same time with the execute method? Otherwise there may be logic error?" Reporter: stack -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] infraio commented on a change in pull request #375: HBASE-21426 TestEncryptionKeyRotation.testCFKeyRotation is flaky
infraio commented on a change in pull request #375: HBASE-21426
TestEncryptionKeyRotation.testCFKeyRotation is flaky
URL: https://github.com/apache/hbase/pull/375#discussion_r303721445
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEncryptionKeyRotation.java
##
@@ -91,6 +92,7 @@ public static void setUp() throws Exception {
conf.setInt("hfile.format.version", 3);
conf.set(HConstants.CRYPTO_KEYPROVIDER_CONF_KEY,
KeyProviderForTesting.class.getName());
conf.set(HConstants.CRYPTO_MASTERKEY_NAME_CONF_KEY, "hbase");
+conf.setInt("hbase.hfile.compaction.discharger.interval", 10 * 60 * 1000);
Review comment:
Why this config change?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22642) Make move operations of RSGroup idempotent
[ https://issues.apache.org/jira/browse/HBASE-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885827#comment-16885827 ] Xiaolin Ha commented on HBASE-22642: Thanks [~xucang]! I have fixed the conflicts and updated the PR. > Make move operations of RSGroup idempotent > -- > > Key: HBASE-22642 > URL: https://issues.apache.org/jira/browse/HBASE-22642 > Project: HBase > Issue Type: Bug > Components: rsgroup >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > Currently, when moving tables or servers to a group, only groupInfo is > checked. And in RSGroup implementation, groupinfo is written to disk before > regions movements are done. If there are some problems caused move regions > abort, some regions will be on wrong regionservers. What's the worse, retry > the move operation will be rejected because of the correct groupinfo. > We think when moving, not only groupInfo should be checked, but also relevant > region assignments should be checked and corrected. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] sunhelly opened a new pull request #387: HBASE-22642 Make move operations of RSGroup idempotent
sunhelly opened a new pull request #387: HBASE-22642 Make move operations of RSGroup idempotent URL: https://github.com/apache/hbase/pull/387 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sunhelly closed pull request #350: Make move operations of RSGroup idempotent
sunhelly closed pull request #350: Make move operations of RSGroup idempotent URL: https://github.com/apache/hbase/pull/350 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #381: HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+
saintstack commented on issue #381: HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+ URL: https://github.com/apache/hbase/pull/381#issuecomment-511653332 Ok. Good by me. Long as other concerns addressed, +1 from me @brfrn169 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-20952) Re-visit the WAL API
[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885803#comment-16885803
]
Hudson commented on HBASE-20952:
Results for branch HBASE-20952
[build #98 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/98/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/98//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/98//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/98//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Re-visit the WAL API
>
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: Josh Elser
>Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup&restore. Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B&R doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[ https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22527: --- Attachment: HBASE-22527.master.002.patch > [hbck2] Add a master web ui to show the problematic regions > --- > > Key: HBASE-22527 > URL: https://issues.apache.org/jira/browse/HBASE-22527 > Project: HBase > Issue Type: Improvement > Components: hbase-operator-tools, hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: 2019-07-16 08-50-07屏幕截图.png, > HBASE-22527.master.001.patch, HBASE-22527.master.002.patch > > > On our cluster which based 2.2.0, we found one problem: there are some opened > regions which had wrong regionserver in meta. The regionserver is not exist. > We used hbck2 to fix them by the following steps. > # disable table > # bypass the stucked close region procedure (as the target regionserver is > not exist) and disable table procedure. > # setRegionState to CLOSED. > # setTableState to DISABLED. > # enable table > We found this problem by scan the hbase:meta. I thought we should add this > feature to hbck2. The we can use hbck2 to find this problem. Thanks. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] Reidddddd commented on issue #386: HBASE-22610 [BucketCache] Rename hbase.offheapcache.minblocksize.
Reidd commented on issue #386: HBASE-22610 [BucketCache] Rename hbase.offheapcache.minblocksize. URL: https://github.com/apache/hbase/pull/386#issuecomment-511643653 Usually, we don't actual remove an old attribute, the way we prefer is marking the old attribute deprecated, then introduce a new one, and LOG warn message and remind them of the new one when user uses the old attribute. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (HBASE-22479) Release sycLatch too early in CreateTable() to get table state failed in postCreateTable
[
https://issues.apache.org/jira/browse/HBASE-22479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885791#comment-16885791
]
Xiaolin Ha edited comment on HBASE-22479 at 7/16/19 2:50 AM:
-
This problem still exists,
{quote}2019-07-16 10:35:22,094 ERROR
[RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=38329]
master.TableStateManager(150): Unable to get table
Group_testCorrectRegionLocationByRetryMoveTables state
org.apache.hadoop.hbase.master.TableStateManager$TableStateNotFoundException:
Group_testCorrectRegionLocationByRetryMoveTables
at
org.apache.hadoop.hbase.master.TableStateManager.getTableState(TableStateManager.java:215)
at
org.apache.hadoop.hbase.master.TableStateManager.isTableState(TableStateManager.java:147)
at
org.apache.hadoop.hbase.master.assignment.AssignmentManager.isTableDisabled(AssignmentManager.java:384)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.lambda$moveTableRegionsToGroup$3(RSGroupAdminServer.java:242)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveRegionsBetweenGroups(RSGroupAdminServer.java:274)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTableRegionsToGroup(RSGroupAdminServer.java:240)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTables(RSGroupAdminServer.java:408)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.assignTableToGroup(RSGroupAdminEndpoint.java:468)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.postCreateTable(RSGroupAdminEndpoint.java:491)
at
org.apache.hadoop.hbase.master.MasterCoprocessorHost$15.call(MasterCoprocessorHost.java:369)
at
org.apache.hadoop.hbase.master.MasterCoprocessorHost$15.call(MasterCoprocessorHost.java:366)
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
at
org.apache.hadoop.hbase.master.MasterCoprocessorHost.postCreateTable(MasterCoprocessorHost.java:366)
at org.apache.hadoop.hbase.master.HMaster$4.run(HMaster.java:2086)
at
org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:2069)
at
org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:667)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:374)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
{quote}
was (Author: xiaolin ha):
This problem still exists,
{quote}2019-07-16 10:35:22,094 ERROR
[RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=38329]
master.TableStateManager(150): Unable to get table
Group_testCorrectRegionLocationByRetryMoveTables state
org.apache.hadoop.hbase.master.TableStateManager$TableStateNotFoundException:
Group_testCorrectRegionLocationByRetryMoveTables
at
org.apache.hadoop.hbase.master.TableStateManager.getTableState(TableStateManager.java:215)
at
org.apache.hadoop.hbase.master.TableStateManager.isTableState(TableStateManager.java:147)
at
org.apache.hadoop.hbase.master.assignment.AssignmentManager.isTableDisabled(AssignmentManager.java:384)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.lambda$moveTableRegionsToGroup$3(RSGroupAdminServer.java:242)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveRegionsBetweenGroups(RSGroupAdminServer.java:274)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTableRegionsToGroup(RSGroupAdminServer.java:240)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTables(RSGroupAdminServer.java:408)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.assignTableToGroup(RSGroupAdminEndpoint.java:468)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.postCreateTable(RSGroupAdminEndpoint.java:491)
at
org.apache.hadoop.hbase.master.MasterCoprocessorHost$15.call(MasterCoprocessorHost.java:369)
at
org.apache.hadoop.hbase.master.MasterCoprocessorHost$15.call(MasterCoprocessorHost.java:366)
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
at
org.apache.hadoop.hbase.master.MasterCoprocessorHost.postCreateTable(MasterCoprocessorHost.java:366)
at org.apache.hadoop.hbase.master.HMaster$4.run(HMaster.java:2086)
at
org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
at org.apache.hadoop.hbase.master.HMaster.createTabl
[jira] [Commented] (HBASE-22479) Release sycLatch too early in CreateTable() to get table state failed in postCreateTable
[
https://issues.apache.org/jira/browse/HBASE-22479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885791#comment-16885791
]
Xiaolin Ha commented on HBASE-22479:
This problem still exists,
{quote}2019-07-16 10:35:22,094 ERROR
[RpcServer.default.FPBQ.Fifo.handler=27,queue=0,port=38329]
master.TableStateManager(150): Unable to get table
Group_testCorrectRegionLocationByRetryMoveTables state
org.apache.hadoop.hbase.master.TableStateManager$TableStateNotFoundException:
Group_testCorrectRegionLocationByRetryMoveTables
at
org.apache.hadoop.hbase.master.TableStateManager.getTableState(TableStateManager.java:215)
at
org.apache.hadoop.hbase.master.TableStateManager.isTableState(TableStateManager.java:147)
at
org.apache.hadoop.hbase.master.assignment.AssignmentManager.isTableDisabled(AssignmentManager.java:384)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.lambda$moveTableRegionsToGroup$3(RSGroupAdminServer.java:242)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveRegionsBetweenGroups(RSGroupAdminServer.java:274)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTableRegionsToGroup(RSGroupAdminServer.java:240)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTables(RSGroupAdminServer.java:408)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.assignTableToGroup(RSGroupAdminEndpoint.java:468)
at
org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.postCreateTable(RSGroupAdminEndpoint.java:491)
at
org.apache.hadoop.hbase.master.MasterCoprocessorHost$15.call(MasterCoprocessorHost.java:369)
at
org.apache.hadoop.hbase.master.MasterCoprocessorHost$15.call(MasterCoprocessorHost.java:366)
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
at
org.apache.hadoop.hbase.master.MasterCoprocessorHost.postCreateTable(MasterCoprocessorHost.java:366)
at org.apache.hadoop.hbase.master.HMaster$4.run(HMaster.java:2086)
at
org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:2069)
at
org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:667)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:374)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
{quote}
> Release sycLatch too early in CreateTable() to get table state failed in
> postCreateTable
>
>
> Key: HBASE-22479
> URL: https://issues.apache.org/jira/browse/HBASE-22479
> Project: HBase
> Issue Type: Bug
> Components: master, rsgroup
>Affects Versions: 2.2.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
>
> SycLatch will be released as soon as prepared create table.
> But if in postCreateTable we need to get some info of the created table, it
> will be failed.
> This can be reproduced by calling createTable() on clusters enabling rsgroup.
> ERROR log is as follows,
> 2019-05-10,11:28:07,394 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=254,queue=14,port=57900]
> org.apache.hadoop.hbase.master.TableStateManager: Unable to get table
> work:error1 state
> org.apache.hadoop.hbase.master.TableStateManager$TableStateNotFoundException:
> work:error1
> at
> org.apache.hadoop.hbase.master.TableStateManager.getTableState(TableStateManager.java:215)
> at
> org.apache.hadoop.hbase.master.TableStateManager.isTableState(TableStateManager.java:147)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.isTableDisabled(AssignmentManager.java:365)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTables(RSGroupAdminServer.java:411)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.assignTableToGroup(RSGroupAdminEndpoint.java:444)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.postCreateTable(RSGroupAdminEndpoint.java:467)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost$13.call(MasterCoprocessorHost.java:351)
> at
> org.apache.hadoop.hbase.master.MasterCoprocessorHost$13.call(MasterCoprocessorHost.java:348)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
> at
> org.apache.hadoop.hbase.cop
[GitHub] [hbase] openinx commented on issue #365: HBASE-22663 The HeapAllocationRatio in WebUI is not accurate because almost all of the heap allocation will happen in another separated allocator name
openinx commented on issue #365: HBASE-22663 The HeapAllocationRatio in WebUI is not accurate because almost all of the heap allocation will happen in another separated allocator named HEAP URL: https://github.com/apache/hbase/pull/365#issuecomment-511636881 Thanks the +1 from @anoopsjohn , I think I forget to update the patch before...say when in disabled reservior case, we can share the global HEAP allocator. Yeah, I've updated the patch(Also some extra UT) Let's see the hadoop QA, if OK I will commit the patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 edited a comment on issue #381: HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+
brfrn169 edited a comment on issue #381: HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+ URL: https://github.com/apache/hbase/pull/381#issuecomment-511632237 Thank you for reviewing @saintstack > The removed methods are only used by OMR? Yes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on issue #381: HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+
brfrn169 commented on issue #381: HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+ URL: https://github.com/apache/hbase/pull/381#issuecomment-511632237 Thank you for reviewing @saintstack > The removed methods are only used by OMR? Yes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on issue #380: HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+
brfrn169 commented on issue #380: HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+ URL: https://github.com/apache/hbase/pull/380#issuecomment-511631980 > Should we squash this PR @brfrn169 given you opened new one sir. Does new one address above comments? Thanks. @saintstack The new one means this PR, right? https://github.com/apache/hbase/pull/381 That is for the master branch. In that PR, I removed OfflineMetaRepair completely whereas in this PR (for branch-2), I changed OfflineMetaRepair to output a message pointing user to hbck2 operator tools. My intention is to deprecate in 2.x and to remove in 3.0. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[ https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22527: --- Attachment: HBASE-22527.master.001.patch > [hbck2] Add a master web ui to show the problematic regions > --- > > Key: HBASE-22527 > URL: https://issues.apache.org/jira/browse/HBASE-22527 > Project: HBase > Issue Type: Improvement > Components: hbase-operator-tools, hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: 2019-07-16 08-50-07屏幕截图.png, HBASE-22527.master.001.patch > > > On our cluster which based 2.2.0, we found one problem: there are some opened > regions which had wrong regionserver in meta. The regionserver is not exist. > We used hbck2 to fix them by the following steps. > # disable table > # bypass the stucked close region procedure (as the target regionserver is > not exist) and disable table procedure. > # setRegionState to CLOSED. > # setTableState to DISABLED. > # enable table > We found this problem by scan the hbase:meta. I thought we should add this > feature to hbck2. The we can use hbck2 to find this problem. Thanks. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[ https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22527: --- Status: Patch Available (was: Open) > [hbck2] Add a master web ui to show the problematic regions > --- > > Key: HBASE-22527 > URL: https://issues.apache.org/jira/browse/HBASE-22527 > Project: HBase > Issue Type: Improvement > Components: hbase-operator-tools, hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: 2019-07-16 08-50-07屏幕截图.png, HBASE-22527.master.001.patch > > > On our cluster which based 2.2.0, we found one problem: there are some opened > regions which had wrong regionserver in meta. The regionserver is not exist. > We used hbck2 to fix them by the following steps. > # disable table > # bypass the stucked close region procedure (as the target regionserver is > not exist) and disable table procedure. > # setRegionState to CLOSED. > # setTableState to DISABLED. > # enable table > We found this problem by scan the hbase:meta. I thought we should add this > feature to hbck2. The we can use hbck2 to find this problem. Thanks. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[ https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885762#comment-16885762 ] Guanghao Zhang commented on HBASE-22527: Pushed a new patch to fix the meta region problem. > [hbck2] Add a master web ui to show the problematic regions > --- > > Key: HBASE-22527 > URL: https://issues.apache.org/jira/browse/HBASE-22527 > Project: HBase > Issue Type: Improvement > Components: hbase-operator-tools, hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: 2019-07-16 08-50-07屏幕截图.png > > > On our cluster which based 2.2.0, we found one problem: there are some opened > regions which had wrong regionserver in meta. The regionserver is not exist. > We used hbck2 to fix them by the following steps. > # disable table > # bypass the stucked close region procedure (as the target regionserver is > not exist) and disable table procedure. > # setRegionState to CLOSED. > # setTableState to DISABLED. > # enable table > We found this problem by scan the hbase:meta. I thought we should add this > feature to hbck2. The we can use hbck2 to find this problem. Thanks. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] brfrn169 commented on issue #380: HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+
brfrn169 commented on issue #380: HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+ URL: https://github.com/apache/hbase/pull/380#issuecomment-511630866 Thank you for reviewing guys. I modified the patch for the review. Could you please review it? @busbey @HorizonNet This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22661) list_regions command in hbase shell is broken
[
https://issues.apache.org/jira/browse/HBASE-22661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885757#comment-16885757
]
Guanghao Zhang commented on HBASE-22661:
+1
> list_regions command in hbase shell is broken
> -
>
> Key: HBASE-22661
> URL: https://issues.apache.org/jira/browse/HBASE-22661
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Toshihiro Suzuki
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-22661.patch
>
>
> I faced the following error in the master branch:
> {code}
> hbase(main):001:0> create "test", "cf"
> 2019-07-07 23:24:15,254 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> Created table test
> Took 6.5678 seconds
> => Hbase::Table - test
> hbase(main):002:0> list_regions "test"
> ERROR: undefined method `getClusterStatus' for
> #
> Did you mean? get_cluster_metrics
> For usage try 'help "list_regions"'
> Took 0.1997 seconds
> {code}
> I didn't check if the other branches have the same issue.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22630) Restore TestReplicationDroppedTables coverage to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885746#comment-16885746 ] Xu Cang commented on HBASE-22630: - ACK, will work on it. > Restore TestReplicationDroppedTables coverage to branch-1 > - > > Key: HBASE-22630 > URL: https://issues.apache.org/jira/browse/HBASE-22630 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Fix For: 1.6.0 > > > TestReplicationDroppedTables was dropped from branch-1. Restore the test > coverage with a test that is not flaky. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HBASE-22695) Store the rsgroup of a table in table configuration
Duo Zhang created HBASE-22695: - Summary: Store the rsgroup of a table in table configuration Key: HBASE-22695 URL: https://issues.apache.org/jira/browse/HBASE-22695 Project: HBase Issue Type: Sub-task Components: rsgroup Reporter: Duo Zhang -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22640) Random init hstore lastFlushTime
[ https://issues.apache.org/jira/browse/HBASE-22640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885744#comment-16885744 ] Xu Cang commented on HBASE-22640: - Nice idea. I'd suggest adding one line comment above your change to explain the rationale. > Random init hstore lastFlushTime > - > > Key: HBASE-22640 > URL: https://issues.apache.org/jira/browse/HBASE-22640 > Project: HBase > Issue Type: Improvement >Reporter: Bing Xiao >Assignee: Bing Xiao >Priority: Major > Fix For: 3.0.0, 2.2.1 > > Attachments: HBASE-22640-master-v1.patch > > > During with open region use current time as hstore last flush time, and no > mush data put cause memstore flush, after flushCheckInterval all memstore > will flush together bring concentrated IO and compaction make high request > latency；So random init lastFlushTime -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22661) list_regions command in hbase shell is broken
[
https://issues.apache.org/jira/browse/HBASE-22661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885745#comment-16885745
]
Duo Zhang commented on HBASE-22661:
---
For the rubocop, the first is not related to the patch, and for the second, it
means we should use single quote instead of double quote, but I just follow the
old way, so I think we could fix them all in a separated issue if we want.
And I think this should go into all branches as at least we should have a test
for list_regions to make sure it does not throw any exceptions.
> list_regions command in hbase shell is broken
> -
>
> Key: HBASE-22661
> URL: https://issues.apache.org/jira/browse/HBASE-22661
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Toshihiro Suzuki
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-22661.patch
>
>
> I faced the following error in the master branch:
> {code}
> hbase(main):001:0> create "test", "cf"
> 2019-07-07 23:24:15,254 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> Created table test
> Took 6.5678 seconds
> => Hbase::Table - test
> hbase(main):002:0> list_regions "test"
> ERROR: undefined method `getClusterStatus' for
> #
> Did you mean? get_cluster_metrics
> For usage try 'help "list_regions"'
> Took 0.1997 seconds
> {code}
> I didn't check if the other branches have the same issue.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22642) Make move operations of RSGroup idempotent
[ https://issues.apache.org/jira/browse/HBASE-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885741#comment-16885741 ] Xu Cang edited comment on HBASE-22642 at 7/16/19 1:03 AM: -- This is an interesting angle to look at this issue. thanks. And I don't see any other overheads beside the one you mentioned. [~Xiaolin Ha] Can you fix the conflict and let hadoopQA run it. was (Author: xucang): this is an interesting angle to look at this issue. thanks. can you elaborate on this comment "repeatedly moving tables/servers to a group might not make regions be moved repeatedly,"? [~Xiaolin Ha] > Make move operations of RSGroup idempotent > -- > > Key: HBASE-22642 > URL: https://issues.apache.org/jira/browse/HBASE-22642 > Project: HBase > Issue Type: Bug > Components: rsgroup >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > Currently, when moving tables or servers to a group, only groupInfo is > checked. And in RSGroup implementation, groupinfo is written to disk before > regions movements are done. If there are some problems caused move regions > abort, some regions will be on wrong regionservers. What's the worse, retry > the move operation will be rejected because of the correct groupinfo. > We think when moving, not only groupInfo should be checked, but also relevant > region assignments should be checked and corrected. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22642) Make move operations of RSGroup idempotent
[ https://issues.apache.org/jira/browse/HBASE-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885741#comment-16885741 ] Xu Cang commented on HBASE-22642: - this is an interesting angle to look at this issue. thanks. can you elaborate on this comment "repeatedly moving tables/servers to a group might not make regions be moved repeatedly,"? [~Xiaolin Ha] > Make move operations of RSGroup idempotent > -- > > Key: HBASE-22642 > URL: https://issues.apache.org/jira/browse/HBASE-22642 > Project: HBase > Issue Type: Bug > Components: rsgroup >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > Currently, when moving tables or servers to a group, only groupInfo is > checked. And in RSGroup implementation, groupinfo is written to disk before > regions movements are done. If there are some problems caused move regions > abort, some regions will be on wrong regionservers. What's the worse, retry > the move operation will be rejected because of the correct groupinfo. > We think when moving, not only groupInfo should be checked, but also relevant > region assignments should be checked and corrected. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22644) if region split fails, the directories of daughterRegions will not be deleted
[ https://issues.apache.org/jira/browse/HBASE-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885734#comment-16885734 ] Xu Cang commented on HBASE-22644: - Agree with [~stack] that we should examine all possibilities that come to this DNRIOE. And at the same time, can you generate a patch and submit patch to this Jira to let HADOOP-QA run? [~Bo Cui] thanks! > if region split fails, the directories of daughterRegions will not be deleted > - > > Key: HBASE-22644 > URL: https://issues.apache.org/jira/browse/HBASE-22644 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Bo Cui >Priority: Critical > Attachments: HBaseFsck.PNG, SplitTransactionImpl.PNG, log.PNG, split > code.PNG > > > if SplitTransactionImpl#createDaughters throws DoNotRetryIOException, and > regionserver is stopping，the directories of daughterRegions will not be > deleted. > !split code.PNG! > the rs log information > !log.PNG! -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[ https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22527: --- Attachment: 2019-07-16 08-50-07屏幕截图.png > [hbck2] Add a master web ui to show the problematic regions > --- > > Key: HBASE-22527 > URL: https://issues.apache.org/jira/browse/HBASE-22527 > Project: HBase > Issue Type: Improvement > Components: hbase-operator-tools, hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: 2019-07-16 08-50-07屏幕截图.png > > > On our cluster which based 2.2.0, we found one problem: there are some opened > regions which had wrong regionserver in meta. The regionserver is not exist. > We used hbck2 to fix them by the following steps. > # disable table > # bypass the stucked close region procedure (as the target regionserver is > not exist) and disable table procedure. > # setRegionState to CLOSED. > # setTableState to DISABLED. > # enable table > We found this problem by scan the hbase:meta. I thought we should add this > feature to hbck2. The we can use hbck2 to find this problem. Thanks. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22650) NPE in AssignmentManager (master crash on startup)
[
https://issues.apache.org/jira/browse/HBASE-22650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885727#comment-16885727
]
Xu Cang commented on HBASE-22650:
-
[~sveyrie] good catch.
Can you please rename patch name to HBASE-22650.branch-1.001.patch and
re-submit by clicking the "submit patch" button to trigger Hadoop-QA. Thanks.
> NPE in AssignmentManager (master crash on startup)
> --
>
> Key: HBASE-22650
> URL: https://issues.apache.org/jira/browse/HBASE-22650
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.2.12, 1.3.5
>Reporter: Sylvain Veyrié
>Priority: Critical
> Labels: patch
> Attachments: AssignmentManager-NPE.patch
>
>
> On HMaster Startup:
>
> {quote}2019-07-02 12:38:11,312 FATAL [orc3:16000.activeMasterManager]
> master.HMaster: Failed to become active master
> java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at
> java.util.concurrent.ConcurrentHashMap.containsKey(ConcurrentHashMap.java:964)
> at
> java.util.concurrent.ConcurrentHashMap$KeySetView.contains(ConcurrentHashMap.java:4558)
> at
> java.util.Collections$UnmodifiableCollection.contains(Collections.java:1032)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:3094)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:495)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:830)
> at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:202)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1883)
> at java.lang.Thread.run(Thread.java:748)
> 2019-07-02 12:38:11,312 FATAL [orc3:16000.activeMasterManager]
> master.HMaster: Master server abort: loaded coprocessors are: []
> 2019-07-02 12:38:11,312 FATAL [orc3:16000.activeMasterManager]
> master.HMaster: Unhandled exception. Starting shutdown.
> java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at
> java.util.concurrent.ConcurrentHashMap.containsKey(ConcurrentHashMap.java:964)
> at
> java.util.concurrent.ConcurrentHashMap$KeySetView.contains(ConcurrentHashMap.java:4558)
> at
> java.util.Collections$UnmodifiableCollection.contains(Collections.java:1032)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:3094)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:495)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:830)
> at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:202)
> at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1883)
> at java.lang.Thread.run(Thread.java:748)
> {quote}
> It happens when regionLocation is null, which may happen just above on line
> 3086 (or as returned by getRegionServer)
> We had this on 1.2.12 with the corresponding patch, but since it is not
> supported anymore, did not submit it.
> Attached is the patch for 1.3.5. Did not test it in 1.4+
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] xcangCRM commented on issue #348: HBASE-22643 : Delete region without archiving only if regiondir is pr…
xcangCRM commented on issue #348: HBASE-22643 : Delete region without archiving only if regiondir is pr… URL: https://github.com/apache/hbase/pull/348#issuecomment-511617881 +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21606) Document use of the meta table load metrics added in HBASE-19722
[
https://issues.apache.org/jira/browse/HBASE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885711#comment-16885711
]
Hudson commented on HBASE-21606:
Results for branch branch-2.2
[build #434 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/434/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/434//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/434//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/434//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Document use of the meta table load metrics added in HBASE-19722
>
>
> Key: HBASE-21606
> URL: https://issues.apache.org/jira/browse/HBASE-21606
> Project: HBase
> Issue Type: Task
> Components: documentation, meta, metrics, Operability
>Affects Versions: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.2, 2.1.3
>Reporter: Sean Busbey
>Assignee: Szalay-Beko Mate
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 1.4.11, 2.1.7
>
> Attachments: HBASE-21606-v1.png
>
>
> HBASE-19722 added a great new tool for figuring out where cluster load is
> coming from. Needs a section in the ref guide
> * When should I use this?
> * Why shouldn't I use it all the time?
> * What does using it look like?
> * How do I use it?
> I think all the needed info for making something to answer these questions is
> in the discussion on HBASE-19722
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22684) The log rolling request maybe canceled immediately in LogRoller due to a race
[
https://issues.apache.org/jira/browse/HBASE-22684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885710#comment-16885710
]
Hudson commented on HBASE-22684:
Results for branch branch-2.2
[build #434 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/434/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/434//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/434//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/434//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The log rolling request maybe canceled immediately in LogRoller due to a race
> -
>
> Key: HBASE-22684
> URL: https://issues.apache.org/jira/browse/HBASE-22684
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: HBASE-22665-UT.patch, HBASE-22684-v1.patch,
> HBASE-22684.patch
>
>
> Copy the comments in HBASE-22681.
> {quote}
> OK, found another problem in LogRoller, where the rollLog flag is set to
> false at the end of the run method, right after we set it to true in the
> logRollRequested method. This cause my test fail after I fixed the
> unackedAppends problem.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22623) Add RegionObserver coprocessor hook for preWALAppend
[ https://issues.apache.org/jira/browse/HBASE-22623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885696#comment-16885696 ] Geoffrey Jacoby commented on HBASE-22623: - Upon looking more closely, it doesn't seem like my idea above of using ObserverContext to pass state to be added to the WALKey will work, because the write path in HRegion doesn't have access to the ObserverContext. So, new plan: there's a MiniBatchOperationInProgress object that gets created at the start of a batch Put or Delete operation (doMiniBatchMutate), and which gets passed to each subsequent coprocessor in the batch mutation pipeline. Both the main pipeline and the coprocs would be able to read or write WALKey attributes to the MiniBatchOperationInProgress, and all that has to change in the main pipeline is an additional optional parameter on doWALAppend (which is private) to instrument the WALKeyImpls it creates. Existing coprocessors won't have to change unless they want to take advantage of the new feature, and there are no new interface methods to implement. > Add RegionObserver coprocessor hook for preWALAppend > > > Key: HBASE-22623 > URL: https://issues.apache.org/jira/browse/HBASE-22623 > Project: HBase > Issue Type: New Feature >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > > While many coprocessor hooks expose the WALEdit to implementing coprocs, > there aren't any that expose the WALKey before it's created and added to the > WALEntry. > It's sometimes useful for coprocessors to be able to edit the WALKey, for > example to add extended attributes using the fields to be added in > HBASE-22622. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-21606) Document use of the meta table load metrics added in HBASE-19722
[
https://issues.apache.org/jira/browse/HBASE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885695#comment-16885695
]
Hudson commented on HBASE-21606:
Results for branch branch-2
[build #2076 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2076/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2076//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2076//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2076//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Document use of the meta table load metrics added in HBASE-19722
>
>
> Key: HBASE-21606
> URL: https://issues.apache.org/jira/browse/HBASE-21606
> Project: HBase
> Issue Type: Task
> Components: documentation, meta, metrics, Operability
>Affects Versions: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.2, 2.1.3
>Reporter: Sean Busbey
>Assignee: Szalay-Beko Mate
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 1.4.11, 2.1.7
>
> Attachments: HBASE-21606-v1.png
>
>
> HBASE-19722 added a great new tool for figuring out where cluster load is
> coming from. Needs a section in the ref guide
> * When should I use this?
> * Why shouldn't I use it all the time?
> * What does using it look like?
> * How do I use it?
> I think all the needed info for making something to answer these questions is
> in the discussion on HBASE-19722
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22684) The log rolling request maybe canceled immediately in LogRoller due to a race
[
https://issues.apache.org/jira/browse/HBASE-22684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885687#comment-16885687
]
Hudson commented on HBASE-22684:
Results for branch branch-2.1
[build #1369 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1369/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1369//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1369//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1369//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The log rolling request maybe canceled immediately in LogRoller due to a race
> -
>
> Key: HBASE-22684
> URL: https://issues.apache.org/jira/browse/HBASE-22684
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: HBASE-22665-UT.patch, HBASE-22684-v1.patch,
> HBASE-22684.patch
>
>
> Copy the comments in HBASE-22681.
> {quote}
> OK, found another problem in LogRoller, where the rollLog flag is set to
> false at the end of the run method, right after we set it to true in the
> logRollRequested method. This cause my test fail after I fixed the
> unackedAppends problem.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-21606) Document use of the meta table load metrics added in HBASE-19722
[
https://issues.apache.org/jira/browse/HBASE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885688#comment-16885688
]
Hudson commented on HBASE-21606:
Results for branch branch-2.1
[build #1369 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1369/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1369//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1369//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1369//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Document use of the meta table load metrics added in HBASE-19722
>
>
> Key: HBASE-21606
> URL: https://issues.apache.org/jira/browse/HBASE-21606
> Project: HBase
> Issue Type: Task
> Components: documentation, meta, metrics, Operability
>Affects Versions: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.2, 2.1.3
>Reporter: Sean Busbey
>Assignee: Szalay-Beko Mate
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 1.4.11, 2.1.7
>
> Attachments: HBASE-21606-v1.png
>
>
> HBASE-19722 added a great new tool for figuring out where cluster load is
> coming from. Needs a section in the ref guide
> * When should I use this?
> * Why shouldn't I use it all the time?
> * What does using it look like?
> * How do I use it?
> I think all the needed info for making something to answer these questions is
> in the discussion on HBASE-19722
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22648) Snapshot TTL
[
https://issues.apache.org/jira/browse/HBASE-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885681#comment-16885681
]
HBase QA commented on HBASE-22648:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
57s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 12m
57s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
4s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 11m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 3m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
8s{color} | {color:red} The patch generated 2 new + 273 unchanged - 1 fixed =
275 total (was 274) {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
1s{color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 9m
6s{color} | {color:blue} patch has no errors when building the reference guide.
See footer for rendered docs, which you should manually inspect. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 0s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
8m 25s{color} | {color:green} the patch passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 12m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{co
[jira] [Commented] (HBASE-22567) HBCK2 addMissingRegionsToMeta
[
https://issues.apache.org/jira/browse/HBASE-22567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885675#comment-16885675
]
stack commented on HBASE-22567:
---
bq. we can invest on this onlineMetaRepair the way it is now, and maybe for
a second version port it to HbckService master side, to keep with hbck2
approach of delegate things to master.
I like the idea of moving OMR to master-side in a v2. Should I commiit
HBASE-22680 then? Thanks.
> HBCK2 addMissingRegionsToMeta
> -
>
> Key: HBASE-22567
> URL: https://issues.apache.org/jira/browse/HBASE-22567
> Project: HBase
> Issue Type: New Feature
> Components: hbck2
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> Following latest discussion on HBASE-21745, this proposes an hbck2 command
> that allows for inserting back regions missing in META that still have
> *regioninfo* available in HDFS. Although this is still an interactive and
> simpler version than the old _OfflineMetaRepair_, it still relies on hdfs
> state as the source of truth, and performs META updates mostly independently
> from Master (apart from requiring Meta table been online).
> For a more detailed explanation on this command behaviour, pasting _command
> usage_ text:
> {noformat}
> To be used for scenarios where some regions may be missing in META,
> but there's still a valid 'regioninfo' metadata file on HDFS.
> This is a lighter version of 'OfflineMetaRepair' tool commonly used for
> similar issues on 1.x release line.
> This command needs META to be online. For each table name passed as
> parameter, it performs a diff between regions available in META,
> against existing regions dirs on HDFS. Then, for region dirs with
> no matches in META, it reads regioninfo metadata file and
> re-creates given region in META. Regions are re-created in 'CLOSED'
> state at META table only, but not in Masters' cache, and are not
> assigned either. A rolling Masters restart, followed by a
> hbck2 'assigns' command with all re-inserted regions is required.
> This hbck2 'assigns' command is printed for user convenience.
> WARNING: To avoid potential region overlapping problems due to ongoing
> splits, this command disables given tables while re-inserting regions.
> An example adding missing regions for tables 'table_1' and 'table_2':
> $ HBCK2 addMissingRegionsInMeta table_1 table_2
> Returns hbck2 'assigns' command with all re-inserted regions.{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22691) [hbase-operator-tools] Move Checkstyle suppression file to different location
[
https://issues.apache.org/jira/browse/HBASE-22691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885668#comment-16885668
]
stack commented on HBASE-22691:
---
Letting this one sit [~Jan Hentschel]. Waiting on another opinion. I'm not sure
build-support is much better than dev-support (it is better yes but there is
probably a better idea out there). Not critical so letting it stew a while.
Thanks.
> [hbase-operator-tools] Move Checkstyle suppression file to different location
> -
>
> Key: HBASE-22691
> URL: https://issues.apache.org/jira/browse/HBASE-22691
> Project: HBase
> Issue Type: Improvement
> Components: hbck2
>Affects Versions: hbck2-1.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Trivial
>
> The Checkstyle suppression file in the {{hbase-operator-tools}} is currently
> defined in the {{dev-support}} directory. It should be moved to a different
> location.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] saintstack commented on a change in pull request #378: HBASE-22684 The log rolling request maybe canceled immediately in Log…
saintstack commented on a change in pull request #378: HBASE-22684 The log
rolling request maybe canceled immediately in Log…
URL: https://github.com/apache/hbase/pull/378#discussion_r303667355
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
##
@@ -211,12 +204,6 @@ public void run() {
} catch (Exception ex) {
LOG.error("Log rolling failed", ex);
abort("Log rolling failed", ex);
- } finally {
-try {
- rollLog.set(false);
-} finally {
- rollLock.unlock();
Review comment:
Great.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #378: HBASE-22684 The log rolling request maybe canceled immediately in Log…
saintstack commented on a change in pull request #378: HBASE-22684 The log
rolling request maybe canceled immediately in Log…
URL: https://github.com/apache/hbase/pull/378#discussion_r303667196
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
##
@@ -55,43 +57,40 @@
@VisibleForTesting
public class LogRoller extends HasThread implements Closeable {
private static final Logger LOG = LoggerFactory.getLogger(LogRoller.class);
- private final ReentrantLock rollLock = new ReentrantLock();
- private final AtomicBoolean rollLog = new AtomicBoolean(false);
- private final ConcurrentHashMap walNeedsRoll = new
ConcurrentHashMap<>();
+ private final ConcurrentMap walNeedsRoll = new
ConcurrentHashMap<>();
private final Server server;
protected final RegionServerServices services;
private volatile long lastrolltime = System.currentTimeMillis();
// Period to roll log.
- private final long rollperiod;
+ private final long rollPeriod;
private final int threadWakeFrequency;
// The interval to check low replication on hlog's pipeline
private long checkLowReplicationInterval;
private volatile boolean running = true;
public void addWAL(final WAL wal) {
-if (null == walNeedsRoll.putIfAbsent(wal, Boolean.FALSE)) {
+if (walNeedsRoll.putIfAbsent(wal, Boolean.FALSE) == null) {
wal.registerWALActionsListener(new WALActionsListener() {
@Override
public void logRollRequested(WALActionsListener.RollRequestReason
reason) {
- walNeedsRoll.put(wal, Boolean.TRUE);
// TODO logs will contend with each other here, replace with e.g.
DelayedQueue
- synchronized(rollLog) {
-rollLog.set(true);
-rollLog.notifyAll();
+ synchronized (LogRoller.this) {
+walNeedsRoll.put(wal, Boolean.TRUE);
+LogRoller.this.notifyAll();
}
}
});
}
}
public void requestRollAll() {
-for (WAL wal : walNeedsRoll.keySet()) {
- walNeedsRoll.put(wal, Boolean.TRUE);
-}
-synchronized(rollLog) {
- rollLog.set(true);
- rollLog.notifyAll();
+synchronized (this) {
+ List wals = new ArrayList(walNeedsRoll.keySet());
+ for (WAL wal : wals) {
+walNeedsRoll.put(wal, Boolean.TRUE);
Review comment:
Ain't the lock checked on each append to the WAL though? (IIRC?)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22537) Split happened Replica region can not be deleted after deleting table successfully and restarting RegionServer
[ https://issues.apache.org/jira/browse/HBASE-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885657#comment-16885657 ] Wellington Chevreuil commented on HBASE-22537: -- Thanks [~sreenivasulureddy]. It seems latest patch has introduced checkstyle issues. Can you address that on a new patch? > Split happened Replica region can not be deleted after deleting table > successfully and restarting RegionServer > -- > > Key: HBASE-22537 > URL: https://issues.apache.org/jira/browse/HBASE-22537 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.1.1 >Reporter: Y. SREENIVASULU REDDY >Assignee: Y. SREENIVASULU REDDY >Priority: Minor > Fix For: 2.1.6 > > Attachments: HBASE-22537.branch-2.1.002.patch, > HBASE-22537.branch-2.1.003.patch, HBASE-22537.branch-2.1.004.patch, > HBASE-22537.branch-2.1.patch > > > [Test step] > 1.create a table (set RegionReplication=2). > 2.insert data to the table utill region be splitted. > 3.Disable and Drop the table. > 4.Parent replica region holding Regionserver, Kill forcefully > 5.HBase WebUI will show that the replica regions will be in RIT. > [Expect Output] > Parent replica region should be deleted. > [Actual Output] > Parent replica region still exists. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22680) [HBCK2] OfflineMetaRepair for hbase2/hbck2
[ https://issues.apache.org/jira/browse/HBASE-22680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885654#comment-16885654 ] stack commented on HBASE-22680: --- Asked up on PR for a +1 on merging [~wchevreuil]... if good by you. > [HBCK2] OfflineMetaRepair for hbase2/hbck2 > -- > > Key: HBASE-22680 > URL: https://issues.apache.org/jira/browse/HBASE-22680 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: hbase-operator-tools-1.0.0 >Reporter: stack >Assignee: stack >Priority: Major > Fix For: hbase-operator-tools-1.0.0 > > > Add tooling and doc that allows recovery of meta after catastrophic loss of > meta table. Like the OfflineMetaRepair tool of hbck1. To address request for > a tool that can 'rebuild meta from region metadata in the filesystem' as > noted in the parent description. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase-operator-tools] saintstack commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2
saintstack commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2 URL: https://github.com/apache/hbase-operator-tools/pull/6#issuecomment-511585697 Ok if I merge this @wchevreuil ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] asf-ci commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2
asf-ci commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2 URL: https://github.com/apache/hbase-operator-tools/pull/6#issuecomment-511584966 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/30/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] wchevreuil commented on a change in pull request #3: Hbase 22567
wchevreuil commented on a change in pull request #3: Hbase 22567
URL: https://github.com/apache/hbase-operator-tools/pull/3#discussion_r303653540
##
File path: hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
##
@@ -334,6 +469,51 @@ private static final String getCommandUsage() {
writer.println(" Returns \"0\" SUCCESS code if it informed region state
is changed, "
+ "\"1\" FAIL code otherwise.");
writer.println();
+writer.println(" " + ADD_MISSING_REGIONS_IN_META_FOR_TABLES + "
...");
+writer.println(" To be used in scenarios where some regions may be
missing in META,");
+writer.println(" but there's still a valid 'regioninfo metadata file on
HDFS. ");
+writer.println(" This is a lighter version of 'OfflineMetaRepair tool
commonly used for ");
+writer.println(" similar issues on 1.x release line. ");
+writer.println(" This command needs META to be online. For each table
name passed as");
+writer.println(" parameter, it performs a diff between regions available
in META, ");
+writer.println(" against existing regions dirs on HDFS. Then, for region
dirs with ");
+writer.println(" no matches in META, it reads regioninfo metadata file
and ");
+writer.println(" re-creates given region in META. Regions are re-created
in 'CLOSED' ");
+writer.println(" state at META table only, but not in Masters' cache,
and are not ");
+writer.println(" assigned either. A rolling Masters restart, followed by
a ");
Review comment:
Good idea! I will submit a separate jira for this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase-operator-tools] wchevreuil commented on issue #3: Hbase 22567
wchevreuil commented on issue #3: Hbase 22567 URL: https://github.com/apache/hbase-operator-tools/pull/3#issuecomment-511584463 Had pushed a new commit d40cfb1, addressing latest suggestions and adding more UTs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] wchevreuil commented on a change in pull request #3: Hbase 22567
wchevreuil commented on a change in pull request #3: Hbase 22567
URL: https://github.com/apache/hbase-operator-tools/pull/3#discussion_r303652863
##
File path: hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
##
@@ -164,6 +178,99 @@ int setRegionState(String region, RegionState.State
newState)
return EXIT_FAILURE;
}
+ Map> reportTablesWithMissingRegionsInMeta(String...
nameSpaceOrTable)
+ throws Exception {
+Map> report;
+try(final MetaFixer metaFixer = new MetaFixer(this.conf)){
+ List names = nameSpaceOrTable != null ?
Arrays.asList(nameSpaceOrTable) : null;
+ report = metaFixer.reportTablesMissingRegions(names);
+} catch (Exception e) {
+ LOG.error("Error reporting missing regions: ", e);
+ throw e;
+}
+if(LOG.isDebugEnabled()) {
+ LOG.debug(formatMissingRegionsInMetaReport(report));
+}
+return report;
+ }
+
+ List addMissingRegionsInMeta(List regionsPath) throws
IOException {
+List reAddedRegionsEncodedNames = new ArrayList<>();
+try(final MetaFixer metaFixer = new MetaFixer(this.conf)){
+ for(Path regionPath : regionsPath){
+metaFixer.putRegionInfoFromHdfsInMeta(regionPath);
+reAddedRegionsEncodedNames.add(regionPath.getName());
+ }
+}
+return reAddedRegionsEncodedNames;
+ }
+
+ Pair, List>
addMissingRegionsInMetaForTables(String...
+ nameSpaceOrTable) {
+ExecutorService executorService = Executors.newFixedThreadPool(
+ nameSpaceOrTable.length > Runtime.getRuntime().availableProcessors() ?
+Runtime.getRuntime().availableProcessors() : nameSpaceOrTable.length);
+List>> futures = new
ArrayList<>(nameSpaceOrTable.length);
+final List readdedRegionNames = new ArrayList<>();
+List executionErrors = new ArrayList<>();
+try(final MetaFixer metaFixer = new MetaFixer(this.conf)){
+ //reducing number of retries in case disable fails due to namespace
table region also missing
+ this.conf.setInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, 1);
+ try(Connection conn = ConnectionFactory.createConnection(this.conf);
+final Admin admin = conn.getAdmin()) {
+Map> report =
reportTablesWithMissingRegionsInMeta(nameSpaceOrTable);
+for (TableName tableName : report.keySet()) {
+ if(admin.tableExists(tableName)) {
+futures.add(executorService.submit(new Callable>() {
+ @Override
+ public List call() throws Exception {
+LOG.debug("running thread for {}",
tableName.getNameWithNamespaceInclAsString());
+try {
+ admin.disableTable(tableName);
+} catch (IOException e) {
+ LOG.debug("Failed to disable table {}, "
+ + "is namespace table also missing regions? Continue
anyway...",
+tableName.getNameWithNamespaceInclAsString(), e);
+}
+List reAddedRegions =
addMissingRegionsInMeta(report.get(tableName));
+try {
+ admin.enableTable(tableName);
+} catch (IOException e) {
+ LOG.debug("Failed enabling table {}. It might be that
namespace table "
+ + "region is also missing.\n"
+ + "After this command finishes, please make sure on this
table state.",
+tableName.getNameWithNamespaceInclAsString(), e);
+}
+return reAddedRegions;
+ }
+}));
+ } else {
+LOG.warn("Table {} does not exist! Skipping...",
+ tableName.getNameWithNamespaceInclAsString());
+ }
+}
+for(Future> f : futures){
+ try {
+readdedRegionNames.addAll(f.get());
+ } catch (ExecutionException e){
+//we want to allow potential running threads to finish, so we
collect execution
+//errors and show those later
+executionErrors.add(e);
+ }
+}
+ }
+} catch (Exception ie){
+ System.out.println("ERROR executing thread: ");
Review comment:
Addressed in PR commit d40cfb1.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase-operator-tools] wchevreuil commented on a change in pull request #3: Hbase 22567
wchevreuil commented on a change in pull request #3: Hbase 22567
URL: https://github.com/apache/hbase-operator-tools/pull/3#discussion_r303652806
##
File path: hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
##
@@ -496,6 +672,51 @@ private static String toString(List things) {
return things.stream().map(i ->
i.toString()).collect(Collectors.joining(", "));
}
+ private String formatMissingRegionsInMetaReport(Map>
report) {
+final StringBuilder builder = new StringBuilder();
+builder.append("Missing Regions for each table:\n\t");
+report.keySet().stream().forEach(table -> {
+ builder.append(table);
+ if (report.get(table).isEmpty()){
+builder.append("->\n\t\t");
+report.get(table).stream().forEach(region ->
builder.append(region.getName())
+ .append(" "));
+ } else {
+builder.append(" -> No missing regions");
+ }
+ builder.append("\n\t");
+});
+return builder.toString();
+ }
+
+ private String formatReAddedRegionsMessage(List readdedRegionNames,
+List executionErrors) {
+final StringBuilder finalText = new StringBuilder();
+finalText.append("Regions re-added into Meta:
").append(readdedRegionNames.size());
+if(readdedRegionNames.isEmpty()){
+ finalText.append("\n")
+.append("WARNING: \n\t")
+.append(readdedRegionNames.size()).append(" regions were added ")
+.append("to META, but these are not yet on Masters cache. \n")
+.append("You need to restart Masters, then run hbck2 'assigns' command
below:\n\t\t")
+.append(buildHbck2AssignsCommand(readdedRegionNames));
+}
+if(executionErrors.isEmpty()){
Review comment:
Yep, this revealed itself as a bug when I last try to use this command.
Addressed in PR commit d40cfb1b22450b03e6d2d4ff38e5e6ea4cb5857e.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase-operator-tools] wchevreuil commented on a change in pull request #3: Hbase 22567
wchevreuil commented on a change in pull request #3: Hbase 22567
URL: https://github.com/apache/hbase-operator-tools/pull/3#discussion_r303652577
##
File path: hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
##
@@ -334,6 +441,58 @@ private static final String getCommandUsage() {
writer.println(" Returns \"0\" SUCCESS code if it informed region state
is changed, "
+ "\"1\" FAIL code otherwise.");
writer.println();
+writer.println(" " + ADD_MISSING_REGIONS_IN_META_FOR_TABLES + "
...");
+writer.println(" To be used in scenarios where some regions may be
missing in META,");
Review comment:
Addressed in PR commit d40cfb1b22450b03e6d2d4ff38e5e6ea4cb5857e.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase-operator-tools] wchevreuil commented on a change in pull request #3: Hbase 22567
wchevreuil commented on a change in pull request #3: Hbase 22567
URL: https://github.com/apache/hbase-operator-tools/pull/3#discussion_r303652344
##
File path: hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
##
@@ -164,6 +178,99 @@ int setRegionState(String region, RegionState.State
newState)
return EXIT_FAILURE;
}
+ Map> reportTablesWithMissingRegionsInMeta(String...
nameSpaceOrTable)
+ throws Exception {
+Map> report;
+try(final MetaFixer metaFixer = new MetaFixer(this.conf)){
+ List names = nameSpaceOrTable != null ?
Arrays.asList(nameSpaceOrTable) : null;
+ report = metaFixer.reportTablesMissingRegions(names);
+} catch (Exception e) {
+ LOG.error("Error reporting missing regions: ", e);
+ throw e;
+}
+if(LOG.isDebugEnabled()) {
+ LOG.debug(formatMissingRegionsInMetaReport(report));
+}
+return report;
+ }
+
+ List addMissingRegionsInMeta(List regionsPath) throws
IOException {
+List reAddedRegionsEncodedNames = new ArrayList<>();
+try(final MetaFixer metaFixer = new MetaFixer(this.conf)){
+ for(Path regionPath : regionsPath){
+metaFixer.putRegionInfoFromHdfsInMeta(regionPath);
+reAddedRegionsEncodedNames.add(regionPath.getName());
+ }
+}
+return reAddedRegionsEncodedNames;
+ }
+
+ Pair, List>
addMissingRegionsInMetaForTables(String...
+ nameSpaceOrTable) {
+ExecutorService executorService = Executors.newFixedThreadPool(
+ nameSpaceOrTable.length > Runtime.getRuntime().availableProcessors() ?
+Runtime.getRuntime().availableProcessors() : nameSpaceOrTable.length);
+List>> futures = new
ArrayList<>(nameSpaceOrTable.length);
+final List readdedRegionNames = new ArrayList<>();
+List executionErrors = new ArrayList<>();
+try(final MetaFixer metaFixer = new MetaFixer(this.conf)){
+ //reducing number of retries in case disable fails due to namespace
table region also missing
+ this.conf.setInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, 1);
+ try(Connection conn = ConnectionFactory.createConnection(this.conf);
+final Admin admin = conn.getAdmin()) {
+Map> report =
reportTablesWithMissingRegionsInMeta(nameSpaceOrTable);
+for (TableName tableName : report.keySet()) {
+ if(admin.tableExists(tableName)) {
+futures.add(executorService.submit(new Callable>() {
+ @Override
+ public List call() throws Exception {
+LOG.debug("running thread for {}",
tableName.getNameWithNamespaceInclAsString());
+try {
+ admin.disableTable(tableName);
+} catch (IOException e) {
+ LOG.debug("Failed to disable table {}, "
+ + "is namespace table also missing regions? Continue
anyway...",
+tableName.getNameWithNamespaceInclAsString(), e);
+}
+List reAddedRegions =
addMissingRegionsInMeta(report.get(tableName));
+try {
+ admin.enableTable(tableName);
+} catch (IOException e) {
+ LOG.debug("Failed enabling table {}. It might be that
namespace table "
+ + "region is also missing.\n"
+ + "After this command finishes, please make sure on this
table state.",
+tableName.getNameWithNamespaceInclAsString(), e);
+}
+return reAddedRegions;
+ }
+}));
+ } else {
+LOG.warn("Table {} does not exist! Skipping...",
+ tableName.getNameWithNamespaceInclAsString());
+ }
+}
+for(Future> f : futures){
+ try {
+readdedRegionNames.addAll(f.get());
+ } catch (ExecutionException e){
+//we want to allow potential running threads to finish, so we
collect execution
Review comment:
Addressed on PR commit d40cfb1b22450b03e6d2d4ff38e5e6ea4cb5857e.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase-operator-tools] wchevreuil commented on a change in pull request #3: Hbase 22567
wchevreuil commented on a change in pull request #3: Hbase 22567 URL: https://github.com/apache/hbase-operator-tools/pull/3#discussion_r303652272 ## File path: hbase-hbck2/pom.xml ## @@ -147,6 +147,11 @@ junit 4.12 + + org.mockito + mockito-core + 2.1.0 Review comment: Addressed on PR commit d40cfb1b22450b03e6d2d4ff38e5e6ea4cb5857e. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] asf-ci commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2
asf-ci commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2 URL: https://github.com/apache/hbase-operator-tools/pull/6#issuecomment-511580985 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/28/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] gjacoby126 commented on a change in pull request #352: HBASE-22622 - WALKey Extended Attributes
gjacoby126 commented on a change in pull request #352: HBASE-22622 - WALKey
Extended Attributes
URL: https://github.com/apache/hbase/pull/352#discussion_r303643171
##
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALKey.java
##
@@ -86,6 +86,21 @@ default long getNonce() {
*/
long getOrigLogSeqNum();
+/**
+ * Return a named String value injected into the WALKey during processing,
such as by a
+ * coprocessor
+ * @param attributeKey The key of a key / value pair
+ */
+ default byte[] getExtendedAttribute(String attributeKey){
+return null;
+ }
+
+/**
+ * Returns a map of all extended attributes injected into this WAL key.
+ */
+ default Map getExtendedAttributes() {
Review comment:
@apurtell - WALKey's interface comments specifically say that setters aren't
permitted. They're meant to be immutable, so the extended attributes will be
set during construction of the WALKeyImpl
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase-operator-tools] asf-ci commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2
asf-ci commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2 URL: https://github.com/apache/hbase-operator-tools/pull/6#issuecomment-511570352 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/27/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21606) Document use of the meta table load metrics added in HBASE-19722
[
https://issues.apache.org/jira/browse/HBASE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885607#comment-16885607
]
Hudson commented on HBASE-21606:
Results for branch master
[build #1226 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1226/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1226//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1226//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1226//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Document use of the meta table load metrics added in HBASE-19722
>
>
> Key: HBASE-21606
> URL: https://issues.apache.org/jira/browse/HBASE-21606
> Project: HBase
> Issue Type: Task
> Components: documentation, meta, metrics, Operability
>Affects Versions: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.2, 2.1.3
>Reporter: Sean Busbey
>Assignee: Szalay-Beko Mate
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 1.4.11, 2.1.7
>
> Attachments: HBASE-21606-v1.png
>
>
> HBASE-19722 added a great new tool for figuring out where cluster load is
> coming from. Needs a section in the ref guide
> * When should I use this?
> * Why shouldn't I use it all the time?
> * What does using it look like?
> * How do I use it?
> I think all the needed info for making something to answer these questions is
> in the discussion on HBASE-19722
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22414) Interruption of moving regions in RSGroup will cause regions on wrong rs
[
https://issues.apache.org/jira/browse/HBASE-22414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885606#comment-16885606
]
Hudson commented on HBASE-22414:
Results for branch branch-2
[build #2075 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2075/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2075//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2075//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2075//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Interruption of moving regions in RSGroup will cause regions on wrong rs
>
>
> Key: HBASE-22414
> URL: https://issues.apache.org/jira/browse/HBASE-22414
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.2.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-22414.master.001.patch
>
>
> We bulk moving regions to target RSGroup, and each movement of region will
> submit a TRSP, but one TRSP encounters exception will make the whole movement
> action terminate. Later regions will not be moved to correct servers unless
> reassign.
> I think we can skip failed moved regions, and retry to move after all has
> been traversed.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22684) The log rolling request maybe canceled immediately in LogRoller due to a race
[
https://issues.apache.org/jira/browse/HBASE-22684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885605#comment-16885605
]
Hudson commented on HBASE-22684:
Results for branch branch-2
[build #2075 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2075/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2075//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2075//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2075//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The log rolling request maybe canceled immediately in LogRoller due to a race
> -
>
> Key: HBASE-22684
> URL: https://issues.apache.org/jira/browse/HBASE-22684
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: HBASE-22665-UT.patch, HBASE-22684-v1.patch,
> HBASE-22684.patch
>
>
> Copy the comments in HBASE-22681.
> {quote}
> OK, found another problem in LogRoller, where the rollLog flag is set to
> false at the end of the run method, right after we set it to true in the
> logRollRequested method. This cause my test fail after I fixed the
> unackedAppends problem.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-21606) Document use of the meta table load metrics added in HBASE-19722
[
https://issues.apache.org/jira/browse/HBASE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885601#comment-16885601
]
Hudson commented on HBASE-21606:
Results for branch branch-2.0
[build #1759 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1759/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1759//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1759//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1759//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Document use of the meta table load metrics added in HBASE-19722
>
>
> Key: HBASE-21606
> URL: https://issues.apache.org/jira/browse/HBASE-21606
> Project: HBase
> Issue Type: Task
> Components: documentation, meta, metrics, Operability
>Affects Versions: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.2, 2.1.3
>Reporter: Sean Busbey
>Assignee: Szalay-Beko Mate
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 1.4.11, 2.1.7
>
> Attachments: HBASE-21606-v1.png
>
>
> HBASE-19722 added a great new tool for figuring out where cluster load is
> coming from. Needs a section in the ref guide
> * When should I use this?
> * Why shouldn't I use it all the time?
> * What does using it look like?
> * How do I use it?
> I think all the needed info for making something to answer these questions is
> in the discussion on HBASE-19722
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22684) The log rolling request maybe canceled immediately in LogRoller due to a race
[
https://issues.apache.org/jira/browse/HBASE-22684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885600#comment-16885600
]
Hudson commented on HBASE-22684:
Results for branch branch-2.0
[build #1759 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1759/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1759//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1759//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1759//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> The log rolling request maybe canceled immediately in LogRoller due to a race
> -
>
> Key: HBASE-22684
> URL: https://issues.apache.org/jira/browse/HBASE-22684
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: HBASE-22665-UT.patch, HBASE-22684-v1.patch,
> HBASE-22684.patch
>
>
> Copy the comments in HBASE-22681.
> {quote}
> OK, found another problem in LogRoller, where the rollLog flag is set to
> false at the end of the run method, right after we set it to true in the
> logRollRequested method. This cause my test fail after I fixed the
> unackedAppends problem.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase-operator-tools] asf-ci commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2
asf-ci commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2 URL: https://github.com/apache/hbase-operator-tools/pull/6#issuecomment-511563385 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/26/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-22694) Use hbase.zookeeper.quorum if fs.hboss.sync.zk.connectionString is not defined
[
https://issues.apache.org/jira/browse/HBASE-22694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser resolved HBASE-22694.
Resolution: Fixed
Hadoop Flags: Reviewed
Tested by hand using the following:
{noformat}
for type in local zk; do for version in 3 2; do mvn clean verify
-Dhadoop.profile=$version -P$type; done; done
{noformat}
Pushed to master.
> Use hbase.zookeeper.quorum if fs.hboss.sync.zk.connectionString is not defined
> --
>
> Key: HBASE-22694
> URL: https://issues.apache.org/jira/browse/HBASE-22694
> Project: HBase
> Issue Type: Improvement
> Components: hboss
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: hbase-filesystem-1.0.0-alpha2
>
>
> In most cases, the ZooKeeper quorum that is used by HBase should be
> sufficient for use by HBoss. Automatically use hbase.zookeeper.quorum if
> fs.hboss.sync.zk.connectionString is not defined.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase-filesystem] joshelser closed pull request #6: HBASE-22694 Fallback to hbase.zookeeper.quorum if fs.hboss.sync.zk.co…
joshelser closed pull request #6: HBASE-22694 Fallback to hbase.zookeeper.quorum if fs.hboss.sync.zk.co… URL: https://github.com/apache/hbase-filesystem/pull/6 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21606) Document use of the meta table load metrics added in HBASE-19722
[
https://issues.apache.org/jira/browse/HBASE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885592#comment-16885592
]
Hudson commented on HBASE-21606:
Results for branch branch-1.4
[build #906 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/906/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/906//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/906//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/906//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Document use of the meta table load metrics added in HBASE-19722
>
>
> Key: HBASE-21606
> URL: https://issues.apache.org/jira/browse/HBASE-21606
> Project: HBase
> Issue Type: Task
> Components: documentation, meta, metrics, Operability
>Affects Versions: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.2, 2.1.3
>Reporter: Sean Busbey
>Assignee: Szalay-Beko Mate
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 1.4.11, 2.1.7
>
> Attachments: HBASE-21606-v1.png
>
>
> HBASE-19722 added a great new tool for figuring out where cluster load is
> coming from. Needs a section in the ref guide
> * When should I use this?
> * Why shouldn't I use it all the time?
> * What does using it look like?
> * How do I use it?
> I think all the needed info for making something to answer these questions is
> in the discussion on HBASE-19722
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-15666) shaded dependencies for hbase-testing-util
[
https://issues.apache.org/jira/browse/HBASE-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885593#comment-16885593
]
Hudson commented on HBASE-15666:
Results for branch branch-1
[build #954 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/954/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/954//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/954//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/954//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> shaded dependencies for hbase-testing-util
> --
>
> Key: HBASE-15666
> URL: https://issues.apache.org/jira/browse/HBASE-15666
> Project: HBase
> Issue Type: New Feature
> Components: test
>Affects Versions: 1.1.0, 1.2.0
>Reporter: Sean Busbey
>Assignee: Balazs Meszaros
>Priority: Critical
> Fix For: 1.5.0, 1.4.11
>
>
> Folks that make use of our shaded client but then want to test things using
> the hbase-testing-util end up getting all of our dependencies again in the
> test scope.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-21606) Document use of the meta table load metrics added in HBASE-19722
[
https://issues.apache.org/jira/browse/HBASE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885594#comment-16885594
]
Hudson commented on HBASE-21606:
Results for branch branch-1
[build #954 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/954/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/954//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/954//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/954//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Document use of the meta table load metrics added in HBASE-19722
>
>
> Key: HBASE-21606
> URL: https://issues.apache.org/jira/browse/HBASE-21606
> Project: HBase
> Issue Type: Task
> Components: documentation, meta, metrics, Operability
>Affects Versions: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.2, 2.1.3
>Reporter: Sean Busbey
>Assignee: Szalay-Beko Mate
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 1.4.11, 2.1.7
>
> Attachments: HBASE-21606-v1.png
>
>
> HBASE-19722 added a great new tool for figuring out where cluster load is
> coming from. Needs a section in the ref guide
> * When should I use this?
> * Why shouldn't I use it all the time?
> * What does using it look like?
> * How do I use it?
> I think all the needed info for making something to answer these questions is
> in the discussion on HBASE-19722
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-15666) shaded dependencies for hbase-testing-util
[
https://issues.apache.org/jira/browse/HBASE-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885591#comment-16885591
]
Hudson commented on HBASE-15666:
Results for branch branch-1.4
[build #906 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/906/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/906//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/906//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/906//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> shaded dependencies for hbase-testing-util
> --
>
> Key: HBASE-15666
> URL: https://issues.apache.org/jira/browse/HBASE-15666
> Project: HBase
> Issue Type: New Feature
> Components: test
>Affects Versions: 1.1.0, 1.2.0
>Reporter: Sean Busbey
>Assignee: Balazs Meszaros
>Priority: Critical
> Fix For: 1.5.0, 1.4.11
>
>
> Folks that make use of our shaded client but then want to test things using
> the hbase-testing-util end up getting all of our dependencies again in the
> test scope.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase-filesystem] joshelser commented on issue #6: HBASE-22694 Fallback to hbase.zookeeper.quorum if fs.hboss.sync.zk.co…
joshelser commented on issue #6: HBASE-22694 Fallback to hbase.zookeeper.quorum if fs.hboss.sync.zk.co… URL: https://github.com/apache/hbase-filesystem/pull/6#issuecomment-511556436 Fast review. Thanks Busbey! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22694) Use hbase.zookeeper.quorum if fs.hboss.sync.zk.connectionString is not defined
[ https://issues.apache.org/jira/browse/HBASE-22694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885561#comment-16885561 ] Josh Elser commented on HBASE-22694: FYI [~wchevreuil], [~busbey] > Use hbase.zookeeper.quorum if fs.hboss.sync.zk.connectionString is not defined > -- > > Key: HBASE-22694 > URL: https://issues.apache.org/jira/browse/HBASE-22694 > Project: HBase > Issue Type: Improvement > Components: hboss >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: hbase-filesystem-1.0.0-alpha2 > > > In most cases, the ZooKeeper quorum that is used by HBase should be > sufficient for use by HBoss. Automatically use hbase.zookeeper.quorum if > fs.hboss.sync.zk.connectionString is not defined. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase-filesystem] joshelser opened a new pull request #6: HBASE-22694 Fallback to hbase.zookeeper.quorum if fs.hboss.sync.zk.co…
joshelser opened a new pull request #6: HBASE-22694 Fallback to hbase.zookeeper.quorum if fs.hboss.sync.zk.co… URL: https://github.com/apache/hbase-filesystem/pull/6 …nnectionString is undefined Simple code change, but some refactoring to add testing for the change. EmbeddedZK was made to be an object which we instantiate, rather than static state and static methods. This lets us re-use the same code for the contract tests without stomping on one-another. Also adds in .gitignore entries for Eclipse metadata. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-22694) Use hbase.zookeeper.quorum if fs.hboss.sync.zk.connectionString is not defined
Josh Elser created HBASE-22694: -- Summary: Use hbase.zookeeper.quorum if fs.hboss.sync.zk.connectionString is not defined Key: HBASE-22694 URL: https://issues.apache.org/jira/browse/HBASE-22694 Project: HBase Issue Type: Improvement Components: hboss Reporter: Josh Elser Assignee: Josh Elser Fix For: hbase-filesystem-1.0.0-alpha2 In most cases, the ZooKeeper quorum that is used by HBase should be sufficient for use by HBoss. Automatically use hbase.zookeeper.quorum if fs.hboss.sync.zk.connectionString is not defined. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase-operator-tools] asf-ci commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2
asf-ci commented on issue #6: HBASE-22680 [HBCK2] OfflineMetaRepair for hbase2/hbck2 URL: https://github.com/apache/hbase-operator-tools/pull/6#issuecomment-511540689 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/25/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL
apurtell commented on a change in pull request #371: HBASE-22648 : Introducing
Snapshot TTL
URL: https://github.com/apache/hbase/pull/371#discussion_r303592367
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
##
@@ -2932,12 +2932,17 @@ public static SnapshotType
createSnapshotType(SnapshotProtos.SnapshotDescription
if (snapshotDesc.getCreationTime() != -1L) {
builder.setCreationTime(snapshotDesc.getCreationTime());
}
+if (snapshotDesc.getTtl() != -1L && snapshotDesc.getTtl() < Long.MAX_VALUE
/ 1000) {
Review comment:
TimeUnit has utility functions for this, better than dividing by constants
as the intent is clear.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL
apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL URL: https://github.com/apache/hbase/pull/371#discussion_r303592755 ## File path: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java ## @@ -1445,6 +1445,9 @@ "hbase.util.default.lossycounting.errorrate"; public static final String NOT_IMPLEMENTED = "Not implemented"; + // Default Snapshot TTL - 30 days (good enough?) + public static final long DEFAULT_SNAPSHOT_TTL = 24 * 3600 * 30; Review comment: The default TTL must be "forever". Anything else violates the principle of least surprise. Use 0 to denote "forever" This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL
apurtell commented on a change in pull request #371: HBASE-22648 : Introducing
Snapshot TTL
URL: https://github.com/apache/hbase/pull/371#discussion_r303594344
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/SnapshotCleanerChore.java
##
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.master.cleaner;
+
+
+import java.io.IOException;
+import java.util.List;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.Stoppable;
+import org.apache.hadoop.hbase.master.snapshot.SnapshotManager;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos;
+
+/**
+ * This chore, every time it runs, will try to delete snapshots that are
expired based on TTL in
+ * seconds configured for each Snapshot
+ */
+@InterfaceAudience.Private
+public class SnapshotCleanerChore extends ScheduledChore {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SnapshotCleanerChore.class);
+ private static final String SNAPSHOT_CLEANER_CHORE_NAME = "SnapshotCleaner";
+ private static final String SNAPSHOT_CLEANER_INTERVAL =
"hbase.master.cleaner.snapshot.interval";
+ private static final int SNAPSHOT_CLEANER_DEFAULT_INTERVAL = 1800 * 1000; //
Default 30 min
+ private static final String SNAPSHOT_CLEANER_DISABLE =
"hbase.master.cleaner.snapshot.disable";
+ private static final String DELETE_SNAPSHOT_EVENT =
+ "Eligible Snapshot for cleanup due to expired TTL.";
+
+ private final SnapshotManager snapshotManager;
+ private final Configuration configuration;
+
+ /**
+ * Construct Snapshot Cleaner Chore with parameterized constructor
+ *
+ * @param stopper When {@link Stoppable#isStopped()} is true, this chore
will cancel and cleanup
+ * @param configuration The configuration to set
+ * @param snapshotManager SnapshotManager instance to manage lifecycle of
snapshot
+ */
+ public SnapshotCleanerChore(Stoppable stopper, Configuration configuration,
+ SnapshotManager snapshotManager) {
+super(SNAPSHOT_CLEANER_CHORE_NAME, stopper,
configuration.getInt(SNAPSHOT_CLEANER_INTERVAL,
+SNAPSHOT_CLEANER_DEFAULT_INTERVAL));
+this.snapshotManager = snapshotManager;
+this.configuration = configuration;
+ }
+
+ @Override
+ protected void chore() {
+final boolean isSnapshotChoreDisabled = this.configuration.getBoolean(
+SNAPSHOT_CLEANER_DISABLE, false);
+if (isSnapshotChoreDisabled) {
+ LOG.debug("Snapshot Cleaner Chore is disabled. Not performing any
cleanup...");
+ return;
+}
+LOG.trace("Snapshot Cleaner Chore is starting up...");
Review comment:
All LOG.debug and LOG.trace invocations should be guarded by if
(LOG.isDebugEnabled()) or if (LOG.isTraceEnabled())
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL
apurtell commented on a change in pull request #371: HBASE-22648 : Introducing
Snapshot TTL
URL: https://github.com/apache/hbase/pull/371#discussion_r303594642
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/SnapshotCleanerChore.java
##
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.master.cleaner;
+
+
+import java.io.IOException;
+import java.util.List;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.Stoppable;
+import org.apache.hadoop.hbase.master.snapshot.SnapshotManager;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos;
+
+/**
+ * This chore, every time it runs, will try to delete snapshots that are
expired based on TTL in
+ * seconds configured for each Snapshot
+ */
+@InterfaceAudience.Private
+public class SnapshotCleanerChore extends ScheduledChore {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SnapshotCleanerChore.class);
+ private static final String SNAPSHOT_CLEANER_CHORE_NAME = "SnapshotCleaner";
+ private static final String SNAPSHOT_CLEANER_INTERVAL =
"hbase.master.cleaner.snapshot.interval";
+ private static final int SNAPSHOT_CLEANER_DEFAULT_INTERVAL = 1800 * 1000; //
Default 30 min
+ private static final String SNAPSHOT_CLEANER_DISABLE =
"hbase.master.cleaner.snapshot.disable";
+ private static final String DELETE_SNAPSHOT_EVENT =
+ "Eligible Snapshot for cleanup due to expired TTL.";
+
+ private final SnapshotManager snapshotManager;
+ private final Configuration configuration;
+
+ /**
+ * Construct Snapshot Cleaner Chore with parameterized constructor
+ *
+ * @param stopper When {@link Stoppable#isStopped()} is true, this chore
will cancel and cleanup
+ * @param configuration The configuration to set
+ * @param snapshotManager SnapshotManager instance to manage lifecycle of
snapshot
+ */
+ public SnapshotCleanerChore(Stoppable stopper, Configuration configuration,
+ SnapshotManager snapshotManager) {
+super(SNAPSHOT_CLEANER_CHORE_NAME, stopper,
configuration.getInt(SNAPSHOT_CLEANER_INTERVAL,
+SNAPSHOT_CLEANER_DEFAULT_INTERVAL));
+this.snapshotManager = snapshotManager;
+this.configuration = configuration;
+ }
+
+ @Override
+ protected void chore() {
+final boolean isSnapshotChoreDisabled = this.configuration.getBoolean(
+SNAPSHOT_CLEANER_DISABLE, false);
+if (isSnapshotChoreDisabled) {
+ LOG.debug("Snapshot Cleaner Chore is disabled. Not performing any
cleanup...");
+ return;
+}
+LOG.trace("Snapshot Cleaner Chore is starting up...");
+try {
+ List completedSnapshotsList =
+ this.snapshotManager.getCompletedSnapshots();
+ for (SnapshotProtos.SnapshotDescription snapshotDescription :
completedSnapshotsList) {
+long snapshotCreatedTime = snapshotDescription.getCreationTime();
+long snapshotTtl = snapshotDescription.getTtl();
+/*
+ * Backward compatibility after the patch deployment on HMaster
+ * Any snapshot with negative or zero ttl should not be deleted
+ * Default ttl value specified by {@HConstants.DEFAULT_SNAPSHOT_TTL}
+ */
+if (snapshotCreatedTime > 0 && snapshotTtl > 0 &&
+snapshotTtl < (Long.MAX_VALUE / 1000)) {
Review comment:
Use TimeUnit for time unit conversions
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL
apurtell commented on a change in pull request #371: HBASE-22648 : Introducing
Snapshot TTL
URL: https://github.com/apache/hbase/pull/371#discussion_r303593588
##
File path: hbase-protocol-shaded/src/main/protobuf/Snapshot.proto
##
@@ -44,6 +44,7 @@ message SnapshotDescription {
optional int32 version = 5;
optional string owner = 6;
optional UsersAndPermissions users_and_permissions = 7;
+ optional int64 ttl = 8 [default = 0];
Review comment:
Here default value is 0. So I guess this answers the question of what should
denote "forever" :-)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL
apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL URL: https://github.com/apache/hbase/pull/371#discussion_r303595006 ## File path: hbase-server/src/main/resources/hbase-webapps/master/snapshot.jsp ## @@ -99,6 +100,7 @@ <% } %> <%= new Date(snapshot.getCreationTime()) %> +<%= snapshot.getTtl() %> Review comment: Can print "FOREVER" here or something similar if value is 0, so the user is less likely to be confused This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL
apurtell commented on a change in pull request #371: HBASE-22648 : Introducing
Snapshot TTL
URL: https://github.com/apache/hbase/pull/371#discussion_r303594585
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/SnapshotCleanerChore.java
##
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.master.cleaner;
+
+
+import java.io.IOException;
+import java.util.List;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.Stoppable;
+import org.apache.hadoop.hbase.master.snapshot.SnapshotManager;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos;
+
+/**
+ * This chore, every time it runs, will try to delete snapshots that are
expired based on TTL in
+ * seconds configured for each Snapshot
+ */
+@InterfaceAudience.Private
+public class SnapshotCleanerChore extends ScheduledChore {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SnapshotCleanerChore.class);
+ private static final String SNAPSHOT_CLEANER_CHORE_NAME = "SnapshotCleaner";
+ private static final String SNAPSHOT_CLEANER_INTERVAL =
"hbase.master.cleaner.snapshot.interval";
+ private static final int SNAPSHOT_CLEANER_DEFAULT_INTERVAL = 1800 * 1000; //
Default 30 min
+ private static final String SNAPSHOT_CLEANER_DISABLE =
"hbase.master.cleaner.snapshot.disable";
+ private static final String DELETE_SNAPSHOT_EVENT =
+ "Eligible Snapshot for cleanup due to expired TTL.";
+
+ private final SnapshotManager snapshotManager;
+ private final Configuration configuration;
+
+ /**
+ * Construct Snapshot Cleaner Chore with parameterized constructor
+ *
+ * @param stopper When {@link Stoppable#isStopped()} is true, this chore
will cancel and cleanup
+ * @param configuration The configuration to set
+ * @param snapshotManager SnapshotManager instance to manage lifecycle of
snapshot
+ */
+ public SnapshotCleanerChore(Stoppable stopper, Configuration configuration,
+ SnapshotManager snapshotManager) {
+super(SNAPSHOT_CLEANER_CHORE_NAME, stopper,
configuration.getInt(SNAPSHOT_CLEANER_INTERVAL,
+SNAPSHOT_CLEANER_DEFAULT_INTERVAL));
+this.snapshotManager = snapshotManager;
+this.configuration = configuration;
+ }
+
+ @Override
+ protected void chore() {
+final boolean isSnapshotChoreDisabled = this.configuration.getBoolean(
+SNAPSHOT_CLEANER_DISABLE, false);
+if (isSnapshotChoreDisabled) {
+ LOG.debug("Snapshot Cleaner Chore is disabled. Not performing any
cleanup...");
+ return;
+}
+LOG.trace("Snapshot Cleaner Chore is starting up...");
+try {
+ List completedSnapshotsList =
+ this.snapshotManager.getCompletedSnapshots();
+ for (SnapshotProtos.SnapshotDescription snapshotDescription :
completedSnapshotsList) {
+long snapshotCreatedTime = snapshotDescription.getCreationTime();
+long snapshotTtl = snapshotDescription.getTtl();
+/*
+ * Backward compatibility after the patch deployment on HMaster
+ * Any snapshot with negative or zero ttl should not be deleted
Review comment:
Again call 0 ttl "forever" and then backwards compat is trivial. This
comment should also be updated to note that 0 TTL is forever.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL
apurtell commented on a change in pull request #371: HBASE-22648 : Introducing
Snapshot TTL
URL: https://github.com/apache/hbase/pull/371#discussion_r303594142
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/SnapshotCleanerChore.java
##
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.master.cleaner;
+
+
+import java.io.IOException;
+import java.util.List;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.Stoppable;
+import org.apache.hadoop.hbase.master.snapshot.SnapshotManager;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos;
+
+/**
+ * This chore, every time it runs, will try to delete snapshots that are
expired based on TTL in
+ * seconds configured for each Snapshot
+ */
+@InterfaceAudience.Private
+public class SnapshotCleanerChore extends ScheduledChore {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SnapshotCleanerChore.class);
+ private static final String SNAPSHOT_CLEANER_CHORE_NAME = "SnapshotCleaner";
+ private static final String SNAPSHOT_CLEANER_INTERVAL =
"hbase.master.cleaner.snapshot.interval";
+ private static final int SNAPSHOT_CLEANER_DEFAULT_INTERVAL = 1800 * 1000; //
Default 30 min
+ private static final String SNAPSHOT_CLEANER_DISABLE =
"hbase.master.cleaner.snapshot.disable";
+ private static final String DELETE_SNAPSHOT_EVENT =
+ "Eligible Snapshot for cleanup due to expired TTL.";
+
+ private final SnapshotManager snapshotManager;
+ private final Configuration configuration;
+
+ /**
+ * Construct Snapshot Cleaner Chore with parameterized constructor
+ *
+ * @param stopper When {@link Stoppable#isStopped()} is true, this chore
will cancel and cleanup
+ * @param configuration The configuration to set
+ * @param snapshotManager SnapshotManager instance to manage lifecycle of
snapshot
+ */
+ public SnapshotCleanerChore(Stoppable stopper, Configuration configuration,
+ SnapshotManager snapshotManager) {
+super(SNAPSHOT_CLEANER_CHORE_NAME, stopper,
configuration.getInt(SNAPSHOT_CLEANER_INTERVAL,
+SNAPSHOT_CLEANER_DEFAULT_INTERVAL));
+this.snapshotManager = snapshotManager;
+this.configuration = configuration;
+ }
+
+ @Override
+ protected void chore() {
+final boolean isSnapshotChoreDisabled = this.configuration.getBoolean(
+SNAPSHOT_CLEANER_DISABLE, false);
Review comment:
If disabled do not create the chore.
There is nothing useful about a chore that wakes up only to log repeatedly
that it will do nothing. :-)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL
apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL URL: https://github.com/apache/hbase/pull/371#discussion_r303593397 ## File path: hbase-common/src/main/resources/hbase-default.xml ## @@ -1864,4 +1864,25 @@ possible configurations would overwhelm and obscure the important. Default is 5 minutes. Make it 30 seconds for tests. See HBASE-19794 for some context. + +hbase.master.cleaner.snapshot.interval +180 + + Snapshot Cleanup chore interval in milliseconds. + The cleanup thread keeps running at this interval + to find all snapshots that are expired based on TTL + and delete them. + + + +hbase.master.snapshot.apply.default.cleaner.ttl +false + + If Snapshot is created without specifying TTL, we can choose to + apply default TTL(30 days). If this config value is set to true, Review comment: Comment does not make sense when default TTL is "forever". See above. Default TTL must be forever. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] apurtell commented on a change in pull request #371: HBASE-22648 : Introducing Snapshot TTL
apurtell commented on a change in pull request #371: HBASE-22648 : Introducing
Snapshot TTL
URL: https://github.com/apache/hbase/pull/371#discussion_r303593699
##
File path: hbase-protocol/src/main/protobuf/HBase.proto
##
@@ -184,6 +184,7 @@ message SnapshotDescription {
optional Type type = 4 [default = FLUSH];
optional int32 version = 5;
optional string owner = 6;
+ optional int64 ttl = 7 [default = 0];
Review comment:
And here the default is 0 too
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22537) Split happened Replica region can not be deleted after deleting table successfully and restarting RegionServer
[
https://issues.apache.org/jira/browse/HBASE-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885537#comment-16885537
]
HBase QA commented on HBASE-22537:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
9s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
6s{color} | {color:red} hbase-server: The patch generated 4 new + 8 unchanged -
0 fixed = 12 total (was 8) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
51s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 10s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.7 2.8.5 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}228m 7s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}281m 20s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAdmin1 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=18.09.7 Server=18.09.7 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/623/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22537 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12974735/HBASE-22537.branch-2.1.004.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 445c4e753b31 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.1 / 1092533308 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/623/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/623/artifact/patchprocess/patch-unit-hbase-
[GitHub] [hbase] Apache-HBase commented on issue #354: HBASE-20368 Fix RIT stuck when a rsgroup has no online servers but AM…
Apache-HBase commented on issue #354: HBASE-20368 Fix RIT stuck when a rsgroup has no online servers but AM… URL: https://github.com/apache/hbase/pull/354#issuecomment-511531296 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 55 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 30 | Maven dependency ordering for branch | | +1 | mvninstall | 270 | master passed | | +1 | compile | 83 | master passed | | +1 | checkstyle | 82 | master passed | | +1 | shadedjars | 280 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 253 | master passed | | +1 | javadoc | 52 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for patch | | +1 | mvninstall | 249 | the patch passed | | +1 | compile | 75 | the patch passed | | +1 | javac | 75 | the patch passed | | +1 | checkstyle | 77 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 271 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 792 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | findbugs | 266 | the patch passed | | +1 | javadoc | 51 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 17710 | hbase-server in the patch failed. | | -1 | unit | 995 | hbase-rsgroup in the patch failed. | | +1 | asflicense | 49 | The patch does not generate ASF License warnings. | | | | 21980 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestCloneSnapshotFromClientNormal | | | hadoop.hbase.replication.TestSyncReplicationMoreLogsInLocalGiveUpSplitting | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.client.TestSnapshotTemporaryDirectory | | | hadoop.hbase.replication.TestSerialSyncReplication | | | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.rsgroup.TestRSGroupsKillRS | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-354/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/354 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e09fe51df46c 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 74731c2a46 | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-354/7/artifact/out/patch-unit-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-354/7/artifact/out/patch-unit-hbase-rsgroup.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-354/7/testReport/ | | Max. process+thread count | 4926 (vs. ulimit of 1) | | modules | C: hbase-server hbase-rsgroup U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-354/7/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22684) The log rolling request maybe canceled immediately in LogRoller due to a race
[
https://issues.apache.org/jira/browse/HBASE-22684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885528#comment-16885528
]
Hudson commented on HBASE-22684:
Results for branch master
[build #1225 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1225/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1225//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1225//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1225//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The log rolling request maybe canceled immediately in LogRoller due to a race
> -
>
> Key: HBASE-22684
> URL: https://issues.apache.org/jira/browse/HBASE-22684
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: HBASE-22665-UT.patch, HBASE-22684-v1.patch,
> HBASE-22684.patch
>
>
> Copy the comments in HBASE-22681.
> {quote}
> OK, found another problem in LogRoller, where the rollLog flag is set to
> false at the end of the run method, right after we set it to true in the
> logRollRequested method. This cause my test fail after I fixed the
> unackedAppends problem.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] srdo commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien…
srdo commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… URL: https://github.com/apache/hbase/pull/361#issuecomment-511529497 The failure on Jenkins seems to be the hbase-server not compiling because it can't find the new ClientTokenUtil class (e.g. https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/5/artifact/out/patch-unit-hbase-server.txt). This build command runs fine for me locally. Is the script maybe set up so it is pulling in an old hbase-client jar? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
