If you don't want to release it you need to not deploy it. That is a pom change unrelated to the staging repo. On Jan 8, 2015 5:37 PM, "Adam Taft" <[email protected]> wrote:
> Benson, > > Quite a bit of NIFI's artifacts are more like application bundles of other > common libraries. Most of the NIFI nars aren't really useful for > application developers to bind to in their own code. These nars are more > part of the application and less useful as a shared common library. > > One of my concerns was accidentally releasing unneeded nars into maven > central. It sounds like the "purgatory" area solves this? Can you speak > to this a little more? > > There's been discussion on whether we care that application nars escape > into maven central - I'm not trying to weigh in on that topic (though, I > think you can read my thoughts between the lines). I'm just asking for > more clarity on the options when using the maven release plugin. Purgatory > might very well allow us to have our cake and eat it too. > > Thanks, > > Adam > > > On Thu, Jan 8, 2015 at 5:07 PM, Benson Margulies <[email protected]> > wrote: > > > On Thu, Jan 8, 2015 at 4:53 PM, Joe Witt <[email protected]> wrote: > > > > > Benson, > > > > > > Well I think we have all the groundwork laid as far as I know for the > > > release plugin. To be honest though (speaking for myself at least) I'm > > not > > > sure what the steps would be that the release plugin would do. I'd > love > > it > > > to be as you describe but am also concerned about how much we'd be > > bugging > > > you to figure that out. I looked at what other projects seem to do and > > it > > > seemed shockingly manual. > > > > > > If you don't mind laying out the steps in more detail when you have > time > > > would love to learn from you on it. I can also try toying around with > > it a > > > bit more off-line. > > > > > > > I don't know what projects you're looking at, but it shouldn't look so > > manual for any project that has a Maven top-level build. > > > > Here's the basic workflow of the maven-release-plugin: > > > > release:prepare: > > > > 1. edit poms to contain 'release version'. > > 2. make a commit. > > 3. make a tag. > > 4. edit poms to contain 'next snapshot' > > 5. make a commit. > > 6. push the whole business (unless you turn that off). > > > > release:perform: > > > > 7. check out from tag in target/checkout (using extra git clone) > > 8. build > > 9. deploy results to target of deploymentManagement > > > > Now comes the next trick. > > > > Apache runs a copy of Sonatype Nexus, that serves as a 'staging > > repository'. So, when step 9 pushes things to the repository, it's > pushing > > them to a special 'purgatory' staging area. That area is available for > > people to look at for testing, but does not propagate along to Maven > > central. > > > > Given all this, then we add one ingredient: an additional maven module > that > > uses the maven-assembly-plugin to build a tarball that satisfies the > > requirements of an Apache source release. This is not the same thing as > > what the maven-source-plugin does at all. Since it will have > > <attach>true</attach>, the resulting tarball swims upstream to the > staging > > repo with everything else. > > > > The vote email points people to that location, and supplies a sha1 for > the > > source ball. Voters download the source release from the staging repo, do > > what they need to do, and vote. Three +1's later, and then you get to > > publish. > > > > Does this help? The only manual steps are really copying to svn to push > to > > the Apache dist area. CXF and all the Maven plugins are examples. > > > > > > > > > > > > > > Thanks > > > Joe > > > > > > On Thu, Jan 8, 2015 at 4:49 PM, Benson Margulies < > [email protected]> > > > wrote: > > > > > > > On Thu, Jan 8, 2015 at 4:41 PM, Joe Witt <[email protected]> wrote: > > > > > > > > > Benson, > > > > > > > > > > Yes that would be much appreciated. > > > > > > > > > > Here is a rough gamplan of what I *think* will be needed to do the > > > > release: > > > > > > > > > > - Branch from develop against a ticket for the overall release > > process > > > > > - Update the version of all the artifacts and dependencies to > > > > > 0.0.1-RC1-incubating or something like that > > > > > - Create the tar.gz of the clean source tree. Sign that and place > > both > > > > the > > > > > signed file and the tar.gz into my people's dir (if i have one of > > > those) > > > > > - Notify folks that this is available so they can pull it and > attempt > > > to > > > > > build from that and validate the signature > > > > > - Once folks agree it is good to merge that to master > > > > > - Create a tar.gz of that and a signed file. Call a vote within > the > > > > team. > > > > > If that is good call a vote within IPMC. > > > > > - IF NO - fix whatever is wrong. > > > > > - If all good then do a maven deploy of the binary artifacts, > source > > > > > bundles, and javadocs > > > > > - Upload the tar.gz of source and the signed file to our dist dir. > > And > > > > > upload convenience binary build tar.gz and zip with sigs to our > dist > > > > > folder. Then create links to these from our downloads page. > > > > > - Update JIRA that the release is closed. > > > > > - Wash/Rinse/Repeat > > > > > > > > > > <note this all is with the understanding that we've done all the > > > > necessary > > > > > review of dependencies, licenses, properly documented them, have > our > > > > > ECCN/encryption stuff properly filed> > > > > > > > > > > > > > Sure, except that I think we can make the maven-release-plugin do a > lot > > > of > > > > the work. > > > > > > > > The idea is that you run the release plugin, and it delivers all > these > > > > goodies to repository.apache.org. Then you call the vote. If the > vote > > > > passes, you (a) push the promote button to push to Maven Central, and > > (b) > > > > check the official source release tarball into the svn area for > > > > distribution. > > > > > > > > I will take a whack at a PR for point (a). > > > > > > > > > > > > > > > > > > > > > > > > > > Might be missing a detail or two but does that sound roughly on the > > > right > > > > > rack? > > > > > > > > > > I had though the maven release plugin would be useful but looks > like > > > > we'll > > > > > just be able to use the versions plugin to update them and can do > the > > > > other > > > > > simple steps manually. > > > > > > > > > > Thanks > > > > > Joe > > > > > > > > > > On Thu, Jan 8, 2015 at 3:46 PM, Benson Margulies < > > > [email protected]> > > > > > wrote: > > > > > > > > > > > Typically people set the maven assembly plugin for this and > include > > > it > > > > in > > > > > > the build. Would you like me to pitch this in? > > > > > > > > > > > > > > > > > > On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <[email protected]> > > > wrote: > > > > > > > > > > > > > On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <[email protected]> > > > wrote: > > > > > > > > > > > > > > > Josh > > > > > > > > > > > > > > > > Awesome. > > > > > > > > > > > > > > > > Honestly I believe we're good on the LICENSE, NOTICE, > > DISCLAIMER, > > > > > > README. > > > > > > > > All dependencies have been reviewed and checked for ASLv2 > > > > > > compatibility. > > > > > > > > > > > > > > > > Will submit the infra ticket now for the dist/ path for keys > > > files > > > > > and > > > > > > > > such. > > > > > > > > > > > > > > > > My guess is that the release artifact should be a tarball of > > all > > > > > > source. > > > > > > > > Could we literally just package up a clean source tree? > Anyone > > > > else > > > > > > have > > > > > > > > views on this? > > > > > > > > > > > > > > > > > > > > > > git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag] is > a > > > > > > perfectly > > > > > > > reasonable thing to do. > > > > > > > > > > > > > > > > > > > > > > > Ideally with this release we'd do it all properly including > > maven > > > > > > > > artifacts, sources/javadocs, and so on. The Maven build does > > > > already > > > > > > > > operate now off a single command at the root to build > > everything > > > > > > > > (build-order is gone) and inherits from the apache parent. > > > > > > > > > > > > > > > > Will need to incorporate RAT. > > > > > > > > > > > > > > > > Thanks for all that - definitely gives some stuff to work on > > and > > > > look > > > > > > > into. > > > > > > > > > > > > > > > > Thanks > > > > > > > > Joe > > > > > > > > > > > > > > > > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > Regardless of what you call it, writing down exactly what > was > > > > done > > > > > to > > > > > > > > make > > > > > > > > > a RC is extremely important early on (I know that I sure > > can't > > > > > > remember > > > > > > > > > what I did last week, much less last release). I'm not sure > > if > > > > > > release > > > > > > > > > guide is formally defined somewhere, but enough information > > > that > > > > > any > > > > > > > > other > > > > > > > > > committer can come by after "you get hit by a train" and > > make a > > > > > > release > > > > > > > > is > > > > > > > > > extremely important. Using the CMS and making a page on the > > > > website > > > > > > for > > > > > > > > > this is extremely easy to do, IMO. > > > > > > > > > > > > > > > > > > Another concern for how to actually make a release is the > > type > > > of > > > > > > > > > packaging that is released. I not talking about the > > > source/binary > > > > > > > release > > > > > > > > > here, literally "what is Apache NiFi 0.0.1-incubating"? Is > > it a > > > > > > > tarball? > > > > > > > > > Are there jars in Maven? etc. A tarball is pretty easy to > > make, > > > > and > > > > > > > > likely > > > > > > > > > the closest to what the build-order.sh script already does > > > (with > > > > > > > Maven's > > > > > > > > > help). I'm not sure if maven-released artifacts are quite > as > > > > > > important > > > > > > > at > > > > > > > > > this point -- if it's not, punting on that will help. > If/when > > > you > > > > > get > > > > > > > to > > > > > > > > > that point, look into the apache parent pom[1]. It is > > extremely > > > > > well > > > > > > > > > automated to use the existing infrastructure to do it all > for > > > > you. > > > > > > > > > > > > > > > > > > Without looking at official documentation (which is me > being > > > > lazy, > > > > > > > > sorry), > > > > > > > > > some other things that will need to be thought about: > > > > > > > > > > > > > > > > > > Start with the releases page [2] > > > > > > > > > > > > > > > > > > * You *must* include a source artifact. It cannot have > > > binaries. > > > > No > > > > > > > Java > > > > > > > > > class files, no jars. A user must be able to download the > > > source > > > > > > > artifact > > > > > > > > > and build it all themselves. Artifacts which include > binaries > > > are > > > > > for > > > > > > > > > user-convenience only and can optionally be provided as > well. > > > > > > > > > > > > > > > > > > * LICENSE, NOTICE and README must all be included in the > top > > > > level > > > > > of > > > > > > > the > > > > > > > > > artifact. The NOTICE is the most painful and must include > any > > > > > > required > > > > > > > > > third-party notices. [3] > > > > > > > > > > > > > > > > > > * You will need to audit your dependencies to make sure > that > > > they > > > > > are > > > > > > > > > ASLv2 compatible [4] > > > > > > > > > > > > > > > > > > * Releases must be cryptographically signed (PGP) [5]. Your > > > > public > > > > > > key > > > > > > > > > should be published to known sites (e.g. pgp.mit.edu) and > > must > > > > be > > > > > > > listed > > > > > > > > > in NiFi's KEYS file [6] (which does not yet exist, probably > > > needs > > > > > an > > > > > > > > infra > > > > > > > > > ticket to create your dist/ directory?). > > > > > > > > > > > > > > > > > > * Verify that every source file contain the proper ASL > > header. > > > > The > > > > > > > > > release-audit-tool[7] (`mvn apache-rat:check`) is a > wonderful > > > > tool > > > > > > that > > > > > > > > can > > > > > > > > > (and should, IMO) be integrated into your build process to > > > > prevent > > > > > > > people > > > > > > > > > from accidentally committing new files between releases > that > > do > > > > not > > > > > > > have > > > > > > > > > the correct header. > > > > > > > > > > > > > > > > > > And suddenly, this is really long :). Short answer: decide > > what > > > > the > > > > > > > > > release should look like (just a tarball?), start vetting > > your > > > > > source > > > > > > > > code > > > > > > > > > for license headers, and looking into NiFi's dependencies > and > > > > their > > > > > > > > > licenses. > > > > > > > > > > > > > > > > > > [1] http://maven.apache.org/pom/asf/ > > > > > > > > > [2] http://www.apache.org/dev/release.html > > > > > > > > > [3] http://www.apache.org/legal/src-headers.html > > > > > > > > > [4] http://www.apache.org/legal/resolved.html#category-x > > > > > > > > > [5] http://www.apache.org/dev/release-signing.html > > > > > > > > > [6] http://www.apache.org/dist/incubator/nifi/KEYS > > > > > > > > > [7] http://creadur.apache.org/rat/ > > > > > > > > > > > > > > > > > > > > > > > > > > > Tony Kurc wrote: > > > > > > > > > > > > > > > > > >> I read the guide Joe linked and a lot of the sticky parts > > are > > > > > marked > > > > > > > > >> "TODO" > > > > > > > > >> and it looks like a work in progress > > > > > > > > >> > > > > > > > > >> "Podlings can short circuit this process by starting out > > with > > > > > > written > > > > > > > > >> release documentation. It is strongly recommended that > > > Podlings > > > > > > invest > > > > > > > > >> time > > > > > > > > >> looking at the best practices recommended in this > document. > > By > > > > > > > selection > > > > > > > > >> and modification, sections of this document can be used to > > > > quickly > > > > > > and > > > > > > > > >> easily bootstrap a release guide. " > > > > > > > > >> > > > > > > > > >> Is step one putting together a release guide? > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt< > [email protected] > > > > > > > > > wrote: > > > > > > > > >> > > > > > > > > >> Hello > > > > > > > > >>> > > > > > > > > >>> Just wanted to stir this one up a bit. Looks like all > > > tickets > > > > > > pegged > > > > > > > > as > > > > > > > > >>> 0.0.1 are resolved (2 remain as of now but seem likely to > > be > > > > > > resolved > > > > > > > > >>> shortly based on their comments). So working through the > > > > release > > > > > > > steps > > > > > > > > >>> available on apache.org and via the link Brock sent. > > > > > > > > >>> > > > > > > > > >>> Anyone interested in this part of the process or who has > > > advice > > > > > to > > > > > > > help > > > > > > > > >>> us > > > > > > > > >>> avoid landmines we're happy to hear it. > > > > > > > > >>> > > > > > > > > >>> Thanks > > > > > > > > >>> Joe > > > > > > > > >>> > > > > > > > > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt< > > [email protected] > > > > > > > > > > > wrote: > > > > > > > > >>> > > > > > > > > >>> Thanks Brock this is very helpful. > > > > > > > > >>>> > > > > > > > > >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland< > > > > > [email protected]> > > > > > > > > >>>> > > > > > > > > >>> wrote: > > > > > > > > >>> > > > > > > > > >>>> Hi, > > > > > > > > >>>>> > > > > > > > > >>>>> This is a decent guide which can be copied: > > > > > > > > >>>>> > > > > > > > > >>>>> http://mrunit.apache.org/pmc/how_to_release.html > > > > > > > > >>>>> > > > > > > > > >>>>> Brock > > > > > > > > >>>>> > > > > > > > > >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt< > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > >>>>> > > > > > > > > >>>>>> Folks, > > > > > > > > >>>>>> > > > > > > > > >>>>>> Looking at the tickets that remain which are presently > > > tied > > > > to > > > > > > > 0.0.1 > > > > > > > > >>>>>> > > > > > > > > >>>>> we're > > > > > > > > >>>>> > > > > > > > > >>>>>> probably 1-2 weeks out from this initial release. Can > > you > > > > > > provide > > > > > > > > >>>>>> > > > > > > > > >>>>> some > > > > > > > > >>> > > > > > > > > >>>> pointers/references or pointers on how to get this ball > > > > rolling > > > > > > and > > > > > > > > >>>>>> > > > > > > > > >>>>> any > > > > > > > > >>> > > > > > > > > >>>> rocks we must move before going down this path? > > > > > > > > >>>>>> > > > > > > > > >>>>>> > > http://incubator.apache.org/guides/releasemanagement.html > > > > > > > > >>>>>> > > > > > > > > >>>>>> That link seems really helpful but has a lot of TODOs > > for > > > > > areas > > > > > > > > which > > > > > > > > >>>>>> > > > > > > > > >>>>> need > > > > > > > > >>>>> > > > > > > > > >>>>>> more explanation. > > > > > > > > >>>>>> > > > > > > > > >>>>>> > > > > > > > > >>>>>> Thanks > > > > > > > > >>>>>> Joe > > > > > > > > >>>>>> > > > > > > > > >>>>>> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
