Heya team, I have created individual pull-requests for each of the major functional pieces outlined in the initial branch. These await review.
I've now started working integrating the test harness into the maven build. After a brief detour for a Yetus plugin, I'm now looking instead at maven integration via exec-maven-plugin. I'm also investigating how to pull the container image build up into maven as well. Thanks, Nic, On Thu, May 4, 2023 at 2:42 PM Nick Dimiduk <ndimi...@apache.org> wrote: > Heya team, > > I've created a feature ticket [0] from which this contribution can hang. > I've created an associated release version [1] and feature branch [2] > against which we can target PRs while things take shape. I've published my > initial extraction of this feature as a whole for your review [3] -- take a > look at the big picture there. For each commit on that branch, I've created > a sub-task on HBASE-27827. Probably reviewers will find other items we need > to peel off as sub-tasks. I'll start turning each of these commits into PRs > suitable for the Apache repo and your perusal. > > I think we're getting due for the 1.3 release of Operator Tools, so I > expect this will land in 1.4. > > Thanks, > Nick > > [0]: https://issues.apache.org/jira/browse/HBASE-27827 > [1]: https://issues.apache.org/jira/browse/HBASE/fixforversion/12353199 > [2]: > https://github.com/apache/hbase-operator-tools/tree/HBASE-27827-kubernetes-deployment > [3]: > https://github.com/apache/hbase-operator-tools/compare/HBASE-27827-kubernetes-deployment...ndimiduk:hbase-operator-tools:HBASE-27827-kubernetes-deployment > > > > On Mon, Mar 13, 2023 at 11:28 AM Nick Dimiduk <ndimi...@apache.org> wrote: > >> Heya team, >> >> Over here at $dayjob, we have an increasing reliance on Kubernetes for >> both development and production workloads. Our tools are maturing and >> we're hoping that they might be of interest to the wider community. >> I'd like to see if there's community interest in receiving some/any of >> them as a contribution. I think we'll also need a plan from ASF Infra >> that makes kubernetes available to us as a project. >> >> We have implemented a basic stack of tools for orchestrating ZK + HDFS >> + HBase on Kubernetes. We use this for running a small local dev >> cluster via MiniKube/KIND ; for ITBLL on smallish distributed clusters >> in a public cloud ; and in production for running clusters of ~100 >> Data Nodes/Region Servers in a public cloud. There was an earlier >> discussion about using our donation of test hardware for running more >> thorough tests in our CI, but one of the limiting factors is full >> cluster deployment. I hope that the community might be interested in >> receiving this tooling as a foundation for more rigorous correctness >> and maybe even performance tests in the open. Furthermore, perhaps the >> wider community has interest in an Apache licensed cluster >> orchestration tool for other uses. >> >> Now for some details: The implementation is built on Kustomize, so >> it's fundamentally transparent resource specification with yaml >> patches for composability; this is in contrast to a solution using >> templates with defined capabilities and interfaces. There is no >> operator ; it's all coordinated via init/bootstrap containers, shell >> scripts, shared volumes for state, &c. For now. >> >> Such a donation will amount to a code drop, which will have its >> challenges. I'm motivated via internal processes to carve it into >> smaller pieces, and I think that will benefit community review as >> well. Perhaps this approach could be used to make the contribution via >> a feature branch. >> >> Is there community interest in adding such a capability to our >> maintained responsibilities? I'd hope that we have several volunteers >> to work with me through the contribution process, and who are >> reasonably confident that they'll be able to help maintain such a >> capability going forward. We'll also need someone who can work with >> Infra to get us access to Kubernetes cluster(s), via whatever means. >> >> What do you think? >> >> Thanks, >> Nick & the HBase team at Apple >> >
