[
https://issues.apache.org/jira/browse/LUCENE-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13965960#comment-13965960
]
Hoss Man commented on LUCENE-5589:
----------------------------------
Robert noted in LUCENE-5590...
bq. ... Our release process generates 600MB of "stuff". The size just keeps
getting bigger and bigger, and is already difficult to work with (e.g. upload
across the internet). ...
While at apachecon the other day, rmuir mentioned his frustration with trying
to publish a 4.7.2 RC over the slow conference network, which led to this
parent issue.
Later, while rmuir wasn't around, sarowe and i were discussing that perhaps the
root problem isn't so much how big *all* of the release artifacts are combined
(since most users aren't downloading more then one of the artifacts per
version) but how to minimize the amount of net transfer involved in preparing &
smoke testing an RC.
This lead me to spit ball the following idea for what an "ideal" release
process might look like by taking more advantage of both jenkins & the svn
"release" repo (which can be used for RCs as well as the final release
artifacts)...
{panel}
* someone steps up to be an RM
* if this is a major or minor release:
** the RM creates the new branch in SVN,
** the RM clicks some buttons in jenkins to create a new "prep-release" job on
the new branch
* a week or so goes by...
** people merge things into the release branch as needed
** the "prep-release" job in jenkins is constantly doing _almost_ everything
the current buildAndPushRelease.py, with the notable exception of signing the
artifacts or pushing the data anywhere
** the "prep-release" job will also build up a directory containing all of the
docs & jdocs we intend to publish on the website for this release once it's
official
* when the RM is ready, they SSH into the lucene jenkins machine, CD into the
latest artifacts dir of the "prep-release" job and:
** run some kind of "stage-maven-jars.py" that handles the maven side of
staging the RC
** svn commits the non-maven artifacts into
https://dist.apache.org/repos/dist/dev/lucene/...
*** NOTE: this is the dir setup by infra specifically for release candidates
*** this could be fully scripted, or there could be a script similar to the way
we do ref-guide publishing that moves files around as needed & then echo's out
hte svn command's to cut/paste for actually committing the files
** run's some sript that stashes away the docs+jdocs for this RC in a more
permanant local dir so they won't be deleted automatically by jenkins
* on the RM's local machine, he runs some "smoke-test-release.py
--no-gpg-check" script pointed at the SVN URL of the release candidate
** this smoke-test-release.pl script will do an svn checkout, followed by all
of the normal things that our existing release smoke checker does
** (the "--no-gpg-check" is because the RM hasn't signed anything yet)
* if the smoke test script passes, then the RM, (on his local machine) runs a
"sign-releases.py" script on the artifacts & svn commits the newly created
*.asc files (or maybe the script does that automatically)
* the RM then sends out the email calling a vote on the RC
** everybody else can run the same "smoke-test-release.py" on the svn RC URL,
but w/o using the "--no-gpg-check" since now the release is signed.
** people cast their votes as normal
* when the vote passes, the RM runs a "publish-rc.py" script, pointing at the
SVN URL for the RC...
** this script will automatically do whatever magic is needed to promote the
"staged" maven artifacts into "real" maven artifacts
** this script will then do a remove "svn mv" from the RC's svn URL to the
final place the release files should live (in
https://dist.apache.org/repos/dist/releases/lucene/...)
*** or, similar to how we deal with the ref guide: maybe it just echo's out the
SVN commands for the RM to cut/paste and run manually
* once the dist mirror network has caught up:
** the RM ssh's back to the jenkins machine and cd's to the directory where the
docs+jdocs for the RC got stashed
** the RM runs some "publish-javadocs.py" script that executes (or echos so the
RM can cut/paste to manually run...) the needed SSH commands to publish the
javadocs ono lucene.apache.org
{panel}
The net gains here would be:
* less manual steps for the RM - jenkins does most of the heavy lifting
* the RM never has to "upload" any release artifacts (or big directories of
javadocs) from their _local_ machine to any remote server - all large transfers
are:
** jenkins -> dist.apache.org
** dist.apache.org -> dist.apache.org (remote svn mv)
** jenkins -> lucene.apache.org
* the RM only has to "download' the RC like everyone else
** the only files the RM "uploads" is the signature files as part of an svn
commit
----
While i certainly agree that it would be nice to find ways to make the
_individual_ release artifacts smaller (to facilitate download time for end
users) perhaps something like this idea would be a good solution for the
_aggregate_ of all the artifacts of a release being large & cumbersome for the
RM?
> release artifacts are too large.
> --------------------------------
>
> Key: LUCENE-5589
> URL: https://issues.apache.org/jira/browse/LUCENE-5589
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
>
> Doing a release currently products *600MB* of artifacts. This is unwieldy...
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
