Sounds good! I'd prefer to run HBCK against 2.2 latest version since that is planned to get the stable pointer soon. It is also fine to run the test for 2.1 and 2.2 HBase versions. Currently there is only a Yetus based pre-commit job for hbase-operator-tools. Similarly to the main HBase repository, the Jenkinsfile can be stored in hbase-operator-tools. In case you need access to builds.a.o for this work you can request access from the PMC.
Peter On Wed, Oct 16, 2019 at 12:02 AM Sakthi <[email protected]> wrote: > I'm planning to start working on a nightly build that can spin up a > mini-cluster, load some data into it, do some actions to bring the cluster > into an undesirable state that hbck2 can fix and then invoke the hbck2 to > see if things work well. > > Plan is to start small with one of the hbck2 commands and remaining ones > can be added incrementally. As of now I would like to start with making > sure the job uses one of the hbase versions (probably 2.1.x), we can > discuss about the need to run the job against all the present hbase > versions/taking in a bunch of hbase versions as input and running against > them/or just a single version. > > The job script would be located in our operator-tools repo. Let me start > digging into creation of a nightly job in the operator-tools (I don't think > we have any as of now). Will create a tracking jira for this. Further > discussions regarding this can be delegated to the jira if deemed more > convenient. > > -Sakthi > > On Mon, Oct 7, 2019 at 11:51 AM Andrew Purtell <[email protected]> > wrote: > > > > We need something different from the chaos monkeys. in this case we're > > not trying to peturb the cluster in ways we think it should handle; we're > > setting up a state we already know requires an outside tool. > > > > Not sure this really falls outside the framework. Add an action that > > invokes hbck. Then, add a policy that schedules the hbck invocation as > part > > of the schedule. That policy would also include the destructive actions > > breaking things in a way the tool needs to fix (or HBase can handle > > intrinsically...) > > > > On Tue, Oct 1, 2019 at 2:11 AM Sean Busbey <[email protected]> wrote: > > > > > I was chatting with Sakthi about automating some testing of hbck2 > > commands. > > > Nothing too fancy, I just want some assurance that they ought to work. > > > > > > This got us talking about how we might purposefully break a cluster to > > meet > > > a set of symptoms that hbck2 knows how to correct. We need something > > > different from the chaos monkeys. in this case we're not trying to > peturb > > > the cluster in ways we think it should handle; we're setting up a state > > we > > > already know requires an outside tool. > > > > > > Where should this kind of tooling live? Main repo next to the monkeys? > > > Alongside hbck2 in operator tools? Somewhere else entirely? > > > > > > > > > -- > > Best regards, > > Andrew > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > decrepit hands > > - A23, Crosstalk > > >
