busbey commented on a change in pull request #32: HBASE-23021
[hbase-operator-tools] README edits in prep for release
URL:
https://github.com/apache/hbase-operator-tools/pull/32#discussion_r324342942
##########
File path: hbase-hbck2/README.md
##########
@@ -519,38 +548,50 @@ command needs to be executed later, so copy and save it
for convenience.
2. For HBase versions prior to 2.3.0, after _addFsRegionsMissingInMeta_
finished successfully and output has been saved,
restart all running HBase Masters.
-3. Once Master's are restarted and META is already online (check if Web UI is
accessible), run
+3. Once Master's are restarted and hbase:meta is already online (check if Web
UI is accessible), run
_assigns_ command from _addFsRegionsMissingInMeta_ output saved per
instructions from #1.
-NOTE: If _namespace_ region is among the missing ones, you will need to add
_--skip_ flag at the
+NOTE: If _namespace_ region is among the missing regions, you will need to add
_--skip_ flag at the
beginning of _assigns_ command returned.
-
-Should a cluster suffer a catastrophic loss of the `hbase:meta` region, a
rough rebuild is possible following the below recipe.
-In outline: stop the cluster; run the _OfflineMetaRepair_ tool which reads
directories and metadata dropped into the filesystem making a best effort at
reconstructing a viable _hbase:meta_ table; restart your cluster; inject an
assign to bring the system namespace table online; and then finally, re-assign
userspace tables you'd like enabled (the rebuilt _hbase:meta_ creates a table
with all tables offline and no regions assigned).
+Should a cluster suffer a catastrophic loss of the `hbase:meta` table, a rough
rebuild is possible using the following recipe.
+In outline, we stop the cluster; run the _HBCK2_ _OfflineMetaRepair_ tool
which reads directories and metadata dropped into the filesystem
+making a best effort at reconstructing a viable _hbase:meta_ table; restart
your cluster; inject an assign to bring the system
+namespace table online; and then finally, re-assign userspace tables you'd
like enabled (the rebuilt _hbase:meta_ creates a table with all tables offline
and no regions assigned).
#### Detailed rebuild recipe
Stop the cluster.
Run the rebuild _hbase:meta_ command from _HBCK2_. This will move aside the
original _hbase:meta_ and put in place a newly rebuilt one. Below is an example
of how to run the tool. It adds the `-details` flag so the tool dumps info on
the regions its found in hdfs:
-```$
HBASE_CLASSPATH_PREFIX=~/checkouts/hbase-operator-tools/hbase-hbck2/target/hbase-hbck2-1.0.0-SNAPSHOT.jar
./bin/hbase org.apache.hbase.hbck1.OfflineMetaRepair -details```
+```
+$
HBASE_CLASSPATH_PREFIX=~/checkouts/hbase-operator-tools/hbase-hbck2/target/hbase-hbck2-1.0.0-SNAPSHOT.jar
./bin/hbase org.apache.hbase.hbck1.OfflineMetaRepair -details
+```
Start the cluster up. It won’t come up fully. It will be stuck because the
_namespace_ table is not online and there is no assign procedure in the
procedure store for this contingency. The hbase master log will show this
state. Here is an example of what it will log:
-```2019-07-10 18:30:51,090 WARN [master/localhost:16000:becomeActiveMaster]
master.HMaster:
hbase:namespace,,1562808216225.725a0fe6c2c869d3d0a9ed82bfa80fa3. is NOT online;
state={725a0fe6c2c869d3d0a9ed82bfa80fa3 state=CLOSED, ts=1562808619952,
server=null}; ServerCrashProcedures=false. Master startup cannot progress, in
holding-pattern until region onlined.```
+```
+2019-07-10 18:30:51,090 WARN [master/localhost:16000:becomeActiveMaster]
master.HMaster:
hbase:namespace,,1562808216225.725a0fe6c2c869d3d0a9ed82bfa80fa3. is NOT online;
state={725a0fe6c2c869d3d0a9ed82bfa80fa3 state=CLOSED, ts=1562808619952,
server=null}; ServerCrashProcedures=false. Master startup cannot progress, in
holding-pattern until region onlined.
+```
To assign the namespace table region, you cannot use the shell. If you use the
shell, it will fail with a `PleaseHoldException` because the master is not yet
up (it is waiting for the namepace table to come online before it declares
itself ‘up’). You have to use the `HBCK2` _assigns_ command. To assign, you
will need the namespace encoded name. It shows in the log quoted above: i.e.
_725a0fe6c2c869d3d0a9ed82bfa80fa3_ in this case. You will also have to pass the
-skip command to ‘skip’ the master version check (without it, your `HBCK2`
invocation will also elicit the above `PleaseHoldException` because the master
is not yet up). Here is an example adding an assign of the namespace table:
-```$
HBASE_CLASSPATH_PREFIX=~/checkouts/hbase-operator-tools/hbase-hbck2/target/hbase-hbck2-1.0.0-SNAPSHOT.jar
./bin/hbase org.apache.hbase.HBCK2 -skip assigns
725a0fe6c2c869d3d0a9ed82bfa80fa3```
+```
+$
HBASE_CLASSPATH_PREFIX=~/checkouts/hbase-operator-tools/hbase-hbck2/target/hbase-hbck2-1.0.0-SNAPSHOT.jar
./bin/hbase org.apache.hbase.HBCK2 -skip assigns
725a0fe6c2c869d3d0a9ed82bfa80fa3
Review comment:
We back to this instead of `hbase hbck -j` ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
