Re: [DISCUSS] operator tools, HBase 3 and StoreFileTracking

I opened HBASE-26826 to consider the backport of SFT into branch-2.5. On Wed, Mar 9, 2022 at 2:47 PM Wellington Chevreuil < wellington.chevre...@gmail.com> wrote: > I would prefer the option 2 suggested by Andrew, SFT backported to 2.5 as > experimental. Regarding the original "thin

Re: [DISCUSS] operator tools, HBase 3 and StoreFileTracking

I would prefer the option 2 suggested by Andrew, SFT backported to 2.5 as experimental. Regarding the original "thin client/server master do all the fix " approach for hbck2 mentioned by Duo, that has already been relaxed to include some logic containing functions for some break scenarios where

Re: [DISCUSS] operator tools, HBase 3 and StoreFileTracking

I just talked about our old design choice, it was not made by me... In fact, for me, I agree that if we want to operate on an active cluster, we'd better go against a procedure. But for having a separated HBCK2 repo outside the main code repo, well, I do not see big advantages and it will

Re: [DISCUSS] operator tools, HBase 3 and StoreFileTracking

I tend to lean towards what Andrew is saying here, but I will also admit that this is in part from not having a good user-experience about getting up an HMaster in maintenance mode to do surgical stuff (feels like two steps instead of just one). Naively, rebuilding the SFT meta files from the

Re: [DISCUSS] operator tools, HBase 3 and StoreFileTracking

There are some recovery cases where the cluster cannot be expected to be up and running. What happens if we have no tooling for those? The user has a dead cluster. So I don't think a requirement that the cluster be up and running always is sufficient. For this type of recovery operator-tools must

Re: [DISCUSS] operator tools, HBase 3 and StoreFileTracking

Sorry a bit late... IIRC, the design of HBCK2 is that, most of the actual fix logic should be done inside hbase(usually as a procedure), and the hbase-operator-tools is just a facade for calling these methods. It will query the cluster to find out which features are supportted. So in general, the

Re: [DISCUSS] operator tools, HBase 3 and StoreFileTracking

Makes sense. Thanks Andrew for clarifying! On Thu, Feb 17, 2022, 21:28 Andrew Purtell wrote: > On Thu, Feb 17, 2022 at 12:19 PM Peter Somogyi > wrote: > > > I like the idea of including the store file tracking in 2.5.0 to unblock > > the HBCK development efforts. > > > > Unfortunately, I was

Re: [DISCUSS] operator tools, HBase 3 and StoreFileTracking

On Thu, Feb 17, 2022 at 12:19 PM Peter Somogyi wrote: > I like the idea of including the store file tracking in 2.5.0 to unblock > the HBCK development efforts. > > Unfortunately, I was not following its development that much. Can it cause > any issues if 2.5.0 has the feature but later an

Re: [DISCUSS] operator tools, HBase 3 and StoreFileTracking

I like the idea of including the store file tracking in 2.5.0 to unblock the HBCK development efforts. Unfortunately, I was not following its development that much. Can it cause any issues if 2.5.0 has the feature but later an incompatible change is needed for SFT? Can it be marked as a beta

Re: [DISCUSS] operator tools, HBase 3 and StoreFileTracking

Another option which I do not see mentioned yet is to extract the relevant common proto and source files from the ‘hbase’ repository into a new repository (‘hbase-storage’?), from which we would release artifacts to be consumed by both hbase and hbase-operator-tools. This maintains D.R.Y.

Re: [DISCUSS] operator tools, HBase 3 and StoreFileTracking

I was talking with Szabolcs prior to him sending this one, and it's a tricky issue for sure. To date, we've solved any HBase API issues by copying code into HBCK2 e.g. HBCKMetaTableAccessor which copies parts of MetaTableAccessor, or we push the logic down server-side to the HBase Master and

[DISCUSS] operator tools, HBase 3 and StoreFileTracking

Hi Folks! While working on adding tools to handle potential FileBased StoreFileTracker issues to HBCK2 (HBASE-26624 ) I ran into multiple problems I'm unsure how to solve. First of all the tools would rely on files not yet available in any of