Just to follow up - as i've progressed on putting together tooling for building artifacts on a regular basis [1], splitting out the packaging tooling seems like the most sane approach. In addition to the reasons already mentioned w.r.t. tagging, it will be much easier to manage isolation of the source tree/tags and the build tooling tree/tags if they are separate.
Jake has gone ahead and created a repo [2]. I will replay our packaging commits into that repo, and send out a patch to remove build-support/packaging from our main sources repo. I will also update relevant docs in the source repo to point to the other repository. Feel free to follow the progress of this in the epic [3] i have for this. [1] https://issues.apache.org/jira/browse/AURORA-851 [2] https://git-wip-us.apache.org/repos/asf/aurora-packaging.git [3] https://issues.apache.org/jira/browse/AURORA-1424 -=Bill On Thu, Jul 30, 2015 at 8:36 AM, Bill Farner <wfar...@apache.org> wrote: > I'm mostly indifferent to whether the packaging code lives in another > repo. Anyone else have an opinion? > > -=Bill > > On Tue, Jul 28, 2015 at 6:19 AM, Jake Farrell <jfarr...@apache.org> wrote: > >> Going back to the 0.9.0 example, if I checkout the 0.9.0 tag it will >> always >> have a broken build-support/packaging/rpm, so even if we had excluded it >> from the source release artifact we generated, apache-aurora-0.9.0.tar.gz, >> it would still be there in the tag and branch and be broken. Anyone >> checking these out would encounter this problem. >> >> Having the packaging in its own repo enables us to have a branch there >> that >> matches up with each source release and we can make any changes on that >> branch and not impact the original source dist's tag or branch. So a rpm >> change or deb change would not change the originally voted on source >> branch. >> >> My concern is with any modifications to our voted on release branches and >> post release having any new commits to them that may, or may not, line up >> with that given initially released artifact. Changes to these branches to >> me feels like a change to the released code which would warrant a new >> release candidate. >> >> -Jake >> >> On Mon, Jul 27, 2015 at 11:33 PM, Bill Farner <wfar...@apache.org> wrote: >> >> > Wouldn't a separate repo make it even more difficult to associate build >> > script changes to source code changes? >> > >> > -=Bill >> > >> > On Mon, Jul 27, 2015 at 8:22 PM, Jake Farrell <jfarr...@apache.org> >> wrote: >> > >> > > In order to make this repeatable it might be good to split this off >> > > entirely into its own aurora-packaging repo. If its still in the main >> > repo >> > > then when we create the release branch the packaging source will >> still be >> > > in the new release branch, but the packaging code would not >> necessarily >> > > line up with that given branched release (like we have currently for >> the >> > > 0.9.0 branch). For people checking out the release branch and not the >> > > source distribution this would be more confusing. >> > > >> > > To eliminate this potential guessing game and expanding on Bill's >> > proposal >> > > I would advocate for us to make the packaging its own repo, >> > > aurora-packaging, and add a set of docker files so we can automate the >> > > build of the given release deb/rpm's and script the process for >> creating >> > > tags for each 0.9.0-1.rpm or 0.9.0-2.deb ... >> > > >> > > -Jake >> > > >> > > On Mon, Jul 27, 2015 at 6:40 PM, Bill Farner <wfar...@apache.org> >> wrote: >> > > >> > > > Hi folks, >> > > > >> > > > An issue that we have not yet officially addressed is release >> > management >> > > as >> > > > it pertains to binary artifacts we produce of Aurora. Today, when >> we >> > > cut a >> > > > release, say 0.9.0, we essentially take a snapshot of (most of) our >> > > > repository as a basis for voting and eventual distribution. >> > > > >> > > > An outstanding question thus far has been how a release is affected >> > when >> > > we >> > > > need to change scripts and configurations for binary distributions >> [1] >> > to >> > > > produce a working binary artifact. By some standards, a bug fix to >> an >> > > RPM >> > > > spec might require another official source release/vote. >> > > > >> > > > I've had several useful discussions with Jake Farrell about this, >> and >> > we >> > > > brought the discussion to the asfinfra hipchat room to hopefully get >> > some >> > > > quick guidance from someone on the ASF board. Please see the quote >> > below >> > > > if you would like to see the transcript. >> > > > >> > > > The summary is that the board allows us to produce binaries as we >> see >> > > fit, >> > > > as the ASF does not consider them official releases. As such, i >> > propose >> > > > that we treat build-support/packaging as distinct from the sources >> we >> > > vote >> > > > on for a source release. I further propose that we omit >> > > > build-support/packaging from our source distributions. This will >> make >> > it >> > > > clear that they are not part of what we are voting on when we cut a >> > > > release. >> > > > >> > > > With this distinction, i would like for us to adopt the practice of >> > > > considering binary distributions 'downstream' from source >> > distributions, >> > > > and bug fixes to packaging do not require a new source distribution >> > > > release. >> > > > >> > > > >> > > > Cheers, >> > > > >> > > > Bill >> > > > >> > > > >> > > > [1] >> > > > >> > > > >> > > >> > >> https://git-wip-us.apache.org/repos/asf?p=aurora.git;a=tree;f=build-support/packaging;h=c77d9b8ab2d8e6ecea2d8028bcdb250239240ffe;hb=HEAD >> > > > >> > > > Jake Farrell 12:03 PM >> > > > > question for the greater audience about packaging, we just cut >> and rc >> > > and >> > > > > voted on it, successfully passed. as part of the src release was >> code >> > > to >> > > > > create deb and rpm packages. went to cut the rpm's to vote on bin >> > > > artifacts >> > > > > and there is a bug in the rpm spec >> > > > > >> > > > > Daniel Gruno (Humbedooh) 12:04 PM >> > > > > burn and reroll? :) >> > > > > >> > > > > Jake Farrell 12:05 PM >> > > > > would the recommendation be to cut a new rc or would having the >> > deb/rpm >> > > > > packaging code in separate repo on its own release be more in >> line. >> > > i.e., >> > > > > package 0.1.0-2.rpm in packaging land >> > > > > >> > > > > Daniel Gruno (Humbedooh) 12:07 PM >> > > > > you could cut a new release and bypass the 72h rule >> > > > > >> > > > > Bill Farner 12:09 PM >> > > > > a goal i'm seeking with this is to decouple source release from >> > binary >> > > > > releases, if possible. that way we don't have things like releases >> > for >> > > N >> > > > > distros that are no-ops because we fixed something in 1. it also >> > means >> > > > that >> > > > > we don't have source releases that have no binary releases because >> > of a >> > > > > packaging spec bug >> > > > > we're currently in the latter situation - we have a perfectly fine >> > > 0.9.0 >> > > > > src release. however, it turns out there's trivial cruft in >> packaging >> > > > specs >> > > > > requiring a post-0.9.0 commit to generate working binaries (key >> > detail >> > > - >> > > > > commit that does not touch the source) >> > > > > >> > > > > Daniel Gruno (Humbedooh) 12:12 PM >> > > > > aren't binaries still viewed as "unofficial convenience"? >> > > > > iow you can do what you like to it, 'cause it ain't ours >> > > > > >> > > > > Tony Stevenson ( pctony ) 12:13 PM >> > > > > AIUI, yes >> > > > > but $AOO etc >> > > > > >> > > > > Daniel Gruno (Humbedooh) 12:13 PM >> > > > > @rbowen you're a board pony, what say you? >> > > > > >> > > > > Tony Stevenson ( pctony ) 12:13 PM >> > > > > neeeighhhhh >> > > > > >> > > > > Daniel Gruno (Humbedooh) 12:13 PM >> > > > > apart from neigh, that is >> > > > > >> > > > > Rich Bowen 12:13 PM >> > > > > Hmm. What? >> > > > > >> > > > > Daniel Gruno (Humbedooh) 12:14 PM >> > > > > bill has a question on ASF binary release policy >> > > > > >> > > > > Bill Farner 12:14 PM >> > > > > details in scrollback, let me know if i've worded poorly :-) >> > > > > + Jake's context a few messages before >> > > > > concrete example: this line in our rpm spec is incorrect >> > > > > >> > > > >> > > >> > >> https://github.com/apache/aurora/blob/master/build-support/packaging/rpm/aurora.spec#L188 >> > > > > fixing it does not alter what i'd consider the source code of the >> > > > project, >> > > > > just the tooling to assemble those sources >> > > > > >> > > > > Daniel Gruno (Humbedooh) 12:18 PM >> > > > > personally, I would favor burning the release and cutting a new >> with >> > a >> > > > > speed vote >> > > > > so that people building from source can generate rpms as well >> > > > > >> > > > > Rich Bowen 12:19 PM >> > > > > I'm not certain what the policy would be in this specific case, >> since >> > > > > binary releases aren't official releases for anybody but OO, but I >> > > would >> > > > > say that if there's a question, you cut another release and >> fasttrack >> > > the >> > > > > vote to put it out there, and eliminate any question. >> > > > > Which ... appears to be what @Humbedooh just said. >> > > > > >> > > > > Daniel Gruno (Humbedooh) 12:20 PM >> > > > > 72 hour rule can be voided (under pain of Greg's fists if you do >> > > > something >> > > > > bad) >> > > > > >> > > > > Bill Farner 12:20 PM >> > > > > so for the end-user, that seems to mean people on different >> distros >> > > could >> > > > > have identical software, but be many releases apart if we had to >> > > iterate >> > > > on >> > > > > one distro's packaging >> > > > > if so, that scenario seems unfortunate for users >> > > > > (or they could be on the same distro for that matter) >> > > > > >> > > > > Daniel Gruno (Humbedooh) 12:21 PM >> > > > > in the end, you can pick whichever solution you want. As @rbowen >> > said, >> > > > > binaries are not officially ours, but yours >> > > > > >> > > > > Bill Farner 12:24 PM >> > > > > that's good to hear. it's probably best that we remove our rpm/deb >> > > > tooling >> > > > > from our source distributions to make the separation more clear >> > > > > thanks for the insight! >> > > > > >> > > > >> > > > -=Bill >> > > > >> > > >> > >> > >
