[ https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572604#comment-17572604 ]
Bryan Beaudreault commented on HBASE-26192: ------------------------------------------- Thanks for the update. Not sure if we'll have time soon too, but just wanted to check. Just while I have you – if someone on my team did pick it up, would you be ok with exposing these metrics over JMX instead of a servlet endpoint? Not sure if that's the way we'd go, it's probably more complicated. But wanted to check if that would suffice. > Master UI hbck should provide a JSON formatted output option > ------------------------------------------------------------ > > Key: HBASE-26192 > URL: https://issues.apache.org/jira/browse/HBASE-26192 > Project: HBase > Issue Type: New Feature > Reporter: Andrew Kyle Purtell > Priority: Minor > Fix For: 2.6.0, 3.0.0-alpha-4 > > Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png > > > It used to be possible to get hbck's verdict of cluster status from the > command line, especially useful for headless deployments, i.e. without > requiring a browser with sufficient connectivity to load a UI, or scrape > information out of raw HTML, or write regex to comb over log4j output. The > hbck tool's output wasn't particularly convenient to parse but it was > straightforward to extract the desired information with a handful of regular > expressions. > HBCK2 has a different design philosophy than the old hbck, which is to serve > as a collection of small and discrete recovery and repair functions, rather > than attempt to be a universal repair tool. This makes a lot of sense and > isn't the issue at hand. Unfortunately the old hbck's utility for reporting > the current cluster health assessment has not been replaced either in whole > or in part. Instead: > {quote} > HBCK2 is for fixes. For listings of inconsistencies or blockages in the > running cluster, you go elsewhere, to the logs and UI of the running cluster > Master. Once an issue has been identified, you use the HBCK2 tool to ask the > Master to effect fixes or to skip-over bad state. Asking the Master to make > the fixes rather than try and effect the repair locally in a fix-it tool's > context is another important difference between HBCK2 and hbck1. > {quote} > Developing custom tooling to mine logs and scrape UI simply to gain a top > level assessment of system health is unsatisfying. There should be a > convenient means for querying the system if issues that rise to the level of > _inconsistency_, in the hbck parlance, are believed to be present. It would > be relatively simple to bring back the experience of invoking a command line > tool to deliver a verdict. This could be added to the hbck2 tool itself but > given that hbase-operator-tools is a separate project an intrinsic solution > is desirable. > An option that immediately comes to mind is modification of the Master's > hbck.jsp page to provide a JSON formatted output option if the HTTP Accept > header asks for text/json. However, looking at the source of hbck.jsp, it > makes more sense to leave it as is and implement a convenient machine > parseable output format elsewhere. This can be trivially accomplished with a > new servlet. Like hbck.jsp the servlet implementation would get a reference > to HbckChore and present the information this class makes available via its > various getters. > The machine parseable output is sufficient to enable headless hbck status > checking but it still would be nice if we could provide operators a command > line tool that formats the information for convenient viewing in a terminal. > That part could be implemented in the hbck2 tool after this proposal is > implemented. -- This message was sent by Atlassian Jira (v8.20.10#820010)