It is great that you brought up this topic Nick! I agree that the optimal solution would be to host all the release related scripts (RC build, verifier) in a common place.
The RC making scrip in hbase-operator-tools that Stack finetuned is meant to work with different artifacts. The current version there gives the RM a smooth experience. It emerged from HBase's create-release script and hopefully it can be used on other releases as well. There are some constraints of the tool like Jira versions should use `hbase-operator-tools` prefix. > This is a good point. I wonder if `dev-support` should be pruned or purged from all branches other than master When we create branch-3 out of master then this becomes a problem again. Would it work if we use a specific branch for such tooling (e.g. dev-tools)? In this case RMs can just clone that branch and don't need the whole HBase repository on their local machine. On Thu, Sep 19, 2019 at 5:40 PM Nick Dimiduk <[email protected]> wrote: > On Wed, Sep 18, 2019 at 10:30 AM Sean Busbey <[email protected]> wrote: > > > If the tooling is in one place where will it live? > > > > As an RM I like not needing to checkout more then the repo I'm trying to > > get a release out on. > > > > My initial thinking was all would be in the main repo, but this is contrary > to your above statement. As I see it, everyone working on HBase has a > checkout of the main repo handy, so for releases spun on developer machines > it's "no big deal." If we can ever get to releases spun through automated > environments, it's all scripted anyway and thus "no big deal." > > In particular my release machine is slow on disk and so updates to the main > > git repo when trying to release something not in the main repo will be > > painful. > > > "slow on disk" as in iops rate or "low on disk" as in capacity? Either way, > I'm surprised to hear about this as a barrier. Is there a side-conversation > we can have about trimming the fat out of our repo(s)? Some git maintenance > that can/needs to be done? I was recently shocked by the girth of our repos > -- nearly 0.5g on the main repo and a whopping 4g on the site! > > For the most part this is also why I usually manually build RCs > > that I run for the main repo, because I can do a shallow clone of the > > release branch instead of needing to get updates to all the active > > branches. > > > > This makes me sad. Automate more, not less! Altering the automation to make > shallow clones of targeted branches should be simple enough, no? > > For testing RCs I guess it's currently all some combination of the > > foundation policies that should be the same and maybe maven profiles? > > > > I agree that codification of foundation policy is the baseline. That would > be enough for me as a first pass. After that, perhaps layers of increasing > sophistication, perhaps repo-dependent. I don't follow your meaning re: > maven profiles. > > Iirc there was already some confusion about using the testing script that > > came in the main repo source bundle vs the one on master. > > > > This is a good point. I wonder if `dev-support` should be pruned or purged > from all branches other than master. Maybe the CI related stuff branches > into it's own directory or directories, and we keep everything else limited > to a single canonical copy on master. This strategy would eliminate > confusion re: what is authority in any given situation but perhaps is too > constraining, given the number of active release branches we maintain. > > What issue are we trying to solve? > > > > Minimize contributor friction. Make it easy for every subscriber to dev@ > to > say "There's a new RC posted and I have an idle machine for an hour / for > the evening / for the weekend, let's just kick the tires." Make it easy for > everyone who's learned the RC tooling on one branch to pinch-hit in on > another branch or another repo. I hear a constant complaint of "scarcity of > volunteer hours" and "I wish we had more people voting in RCs" and "I wish > we could keep a monthly release cadence on every branch we're maintaining". > So I'd like to see a focused effort on maximizing the volunteer human and > machine time that's thrown our way. > > Thanks, > Nick > > On Wed, Sep 18, 2019, 11:50 Nick Dimiduk <[email protected]> wrote: > > > > > Heya, > > > > > > Looks like we have quite a few repos now, each of which must produce > > > artifacts that follow the Apache protocols. I also see we have some > nice > > > tools built up in dev-support in the main repo for RM's who build > release > > > candidates and community members to vote on them. I tried to use our > > > hbase-vote.sh on this operator-tools RC and found it mostly works. I > > think > > > a few adjustments on each end would allow these tools to work across > > repos, > > > so I filed HBASE-23048. However, I now see that operator-tools has its > > own > > > dev-support/create-release, so I wonder what direction we want to take > > with > > > this automation, so here I come to the list. > > > > > > Do we want to have independent tooling for each repo? Are the processes > > of > > > building RC's different enough to warrant separate tools? Is a single > > tool > > > that can build an RC for all of them not worth the trouble? At the very > > > least, I think the hbase-vote.sh can be made to work with releases > > > generated from each of the repos, as it's not doing all that much. > > > > > > Thoughts? > > > > > > Thanks, > > > Nick > > > > > > On Wed, Sep 18, 2019 at 9:42 AM Nick Dimiduk <[email protected]> > > wrote: > > > > > > > +1 > > > > > > > > - Verified signatures. > > > > - Verified checksums. > > > > - Built from source tarball successfully. > > > > - Ran unit tests from source tarball, pass. > > > > - Ran rat check from source tarball, pass. > > > > > > > > > > > > On Mon, Sep 16, 2019 at 11:31 AM Peter Somogyi <[email protected]> > > > > wrote: > > > > > > > >> Please vote on this Apache HBase Operator Tools Release Candidate > > (RC), > > > >> 1.0.0. > > > >> > > > >> The VOTE will remain open for at least 72 hours. > > > >> > > > >> [ ] +1 Release this package as Apache HBase Operator Tools 1.0.0 > > > >> [ ] -1 Do not release this package because ... > > > >> > > > >> The tag to be voted on is 1.0.0RC0: > > > >> > > > >> https://github.com/apache/hbase-operator-tools/tree/1.0.0RC0 > > > >> > > > >> The release files, including signatures, digests, as well as > > CHANGES.md > > > >> and RELEASENOTES.md included in this RC can be found at: > > > >> > > > >> https://dist.apache.org/repos/dist/dev/hbase/1.0.0RC0/ > > > >> > > > >> Maven artifacts are available in a staging repository at: > > > >> > > > >> > > > > https://repository.apache.org/content/repositories/orgapachehbase-1348/ > > > >> > > > >> Artifacts were signed with the [email protected] key which can be > > > found > > > >> in: > > > >> > > > >> https://dist.apache.org/repos/dist/release/hbase/KEYS > > > >> > > > >> To learn more about Apache HBase Operator Tools, please see > > > >> http://hbase.apache.org/ > > > >> > > > >> Thanks, > > > >> Your HBase Release Manager > > > >> > > > > > > > > > >
