On Wed, Jun 26, 2019 at 5:29 PM leerho <[email protected]> wrote: > Kenn, > > I have been studying the structure and content of the repositories: > > - DIST: > - DEV: dist.apache.org/repos/dist/dev > - RELEASE: dist.apache.org/repos/dist/release (Raw) == > www.apache.org/dist (html, autogenerated) > - http://archive.apache.org/dist/ (html, autogenerated, Release > only) > - GitBox: https://gitbox.apache.org/repos/asf > - Nexus: https://repository.apache.org > - Repositories / Releases: > https://repository.apache.org/content/repositories/releases/ > - Repositories / Snapshots: > https://repository.apache.org/content/repositories/snapshots/ > > Observations: (please correct me if I am wrong :) ) > > DIST sites: > The only way I know how to upload content to these sites is using SVN. > (Correct?) > > 1) It appears that DIST/DEV is pretty much whatever we want to put > there. There is large variance and inconsistent content across different > projects. Its only purpose that I can see would be to test the ability to > write content to DIST prior to putting it into DIST/RELEASE. (Correct?) >
dist/dev is where you put things prior to release, so they can be voted upon. Here's an example vote thread beginning for Beam: https://lists.apache.org/thread.html/2d3378dd7d95cafc60fa9c44b3b722cad7f2f829a8db275ce0279983@%3Cdev.beam.apache.org%3E dist/release is where you move them after approval in a vote thread 2) The content placed in DIST/RELEASE appears to be Zips of site GitHub > branches and signing files. Relatively recent releases only. Whatever is > uploaded here is automatically copied to archive. So I presume only > approved releases go here. Incubation Releases are OK, but not SNAPSHOTS. > I'm not sure I see the usefulness of this repo except for the historical > record. If I want a download of a specific release I would go to the > GitHub site and select the branch or tag I want and then fork the site. > Nonetheless, this is the official record of releases for Apache. > Yes, only specific releases. Note that "release" in this case means an ASF source release. It should be everything necessary to take the whole project, build it, build a derivative work, etc, etc. So a zip of the whole archive is convenient. You can set up the maven source release to collect all the files you want, but IMO it is a bit pointless to do that since GitHub already does what you need. 3) The GitBox site is automatically created and updated from the > http://GitHub.com/apache/ repositories. It appears to be just a summary > of commits. Are these mirrored? Where? > This is just a UI issue. The commits are there in gitbox. In fact, it is really the authoritative copy though two-way sync makes that kind of irrelevant. Example: https://gitbox.apache.org/repos/asf?p=incubator-datasketches.git;a=commitdiff;h=13294043b4e5f4b0ae9532ca08d1e37dab60bf79 > > 4) Nexus: This is optional and the internal project structures are more > complex. All the individual, Maven-addressable dependency elements are > archived here. Apparently whether a Maven generated "release" ends up in > Snapshots or in Releases depends strictly on how the release version tag > contains "SNAPSHOT" or not. (Is this correct?). > Actually, no. They are separate URLS that the maven release publishing plugin can be configured to point to. There may be some magic in the plugin that decides to point at one or the other, but I thought it was manual. > **** > I have been studying your Apache Beam Release Guide as of Mar 17, 2018 > <https://github.com/apache/beam/blob/16106263a1c10afae7b44e5abb8be1ebddd799f0/website/src/contribute/release-guide.md>, > which (I think) just before you started transitioning to Gradle. > > 1) The point I am stuck on is "Create a new version in JIRA". I can't > find out where to do that. I have filed a ticket with INFRA for this. > Since you prefer GitHub issues, this is not necessary. Beam uses this to set the "Fix Version" field which serves both as a burndown for release-blocking issues and then afterwards the release notes are automatic. But it is extra and has no real necessity. > Your sharing these Release Guides are *far far* more helpful than what I > can get from the Apache documentation! Thank you! > Sounds like I/we should take some time writing this stuff up. I wonder if general@incubator or training@apache or dev@community would be good venues for figuring out a good home. Kenn > Nonetheless, I could still use some 1:1 help from a release engineer > experienced with Apache/Maven release process. Is there someone on your > team that could give me a hand? I don't think it would take more than an > hour. Our project is not nearly as complex as Beam. > > Cheers, > > Lee. > > > > > > > > > > > > > On Thu, Jun 20, 2019 at 10:17 PM Kenneth Knowles <[email protected]> wrote: > >> I think a lot of these questions are good for general@incubator. I >> expect that podlings put their releases under some incubator-specific >> directory on dist.apache.org and that is why it looks different than it >> does for Beam. >> >> As for release manager, typically a thread to discuss releasing is on dev@ >> and that establishes the release manager. Officially, I think whoever calls >> a [VOTE] with a release candidate is the manager. Or maybe there's no >> official definition at all. >> >> Kenn >> >> On Thu, Jun 20, 2019 at 5:29 PM leerho <[email protected]> wrote: >> >>> Kenn, >>> >>> I know you are super busy with your new house and move, so I feel bad >>> about bugging you. So I am wondering if we could set up a video conf >>> whenever you are free to go over this stuff. >>> >>> I can't make much progress until, I can get access to some of the key >>> resources: dist.apache.org, and setting the right stuff in JIRA. >>> >>> Would designating me as "Release Manager" help ?? >>> >>> Cheers, >>> >>> Lee. >>> >>> On Wed, Jun 19, 2019 at 6:56 PM Kenneth Knowles <[email protected]> wrote: >>> >>>> TBH I did not personally do any incubating releases. So there might be >>>> something funky there, since the only "real" PMC is Incubator PMC. >>>> >>>> 1) I will check the logs to see how and when our release managers got >>>> added. >>>> >>>> 2) You only need an account on pypi to release Python "convenience >>>> binaries" to pypi. >>>> >>>> 3) How you manage versioning in Jira is your business. I'm not sure how >>>> podling administration of Jira works. All the initial PMC members should >>>> probably be able to manipulate this. It is just a bucket for bugs. >>>> >>>> 3a) About the coordinates: >>>> https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-core. >>>> You can see that the main package is the same for pre- and post-incubating >>>> releases. You don't "release" RCs; they can be accessed via the staging >>>> repository before finalization. >>>> >>>> 4) We used to use maven and the maven release plugin. It is designed in >>>> a world without branches. All of those steps are useless and not necessary >>>> in a git world. You just make a branch and edit the version in your pom on >>>> that branch. >>>> >>>> Kenn >>>> >>>> On Wed, Jun 19, 2019 at 6:50 PM leerho <[email protected]> wrote: >>>> >>>>> Kenn, >>>>> >>>>> Following your Beam Release Guide: >>>>> >>>>> OK, I got new GPG keys generated, >>>>> >>>>> 1) but now I need to add them to dist.apache.org. >>>>> >>>>> There is no entry for datasketches in the list, and I have no write >>>>> access to that site. >>>>> >>>>> Apparently only PMC members can do this, so I think that means you :) >>>>> >>>>> Please add the --global option. >>>>> >>>>> my GPG Key ID is 05F2FA8F8CD4A902 >>>>> and the fingerprint is : 82DF 4C90 F72D 57E3 6CB8 355B 05F2 FA8F 8CD4 >>>>> A902 >>>>> I will need it entered into both dev/ and release/ KEYS files. >>>>> >>>>> 2) Do I need an account on PyPl ?? Is "Release Manager" is Beam >>>>> specific?? and Gradle specific ? (It seems to be so) >>>>> >>>>> 3) Create a new version in JIRA : Only PMC members can do this ... >>>>> I will need to create a release Item for the upcoming release ... >>>>> I need to release *incubator-sketches-memory* The new version will >>>>> be "1.0.0-incubating-memory-rc1" (I think I add the rc1 here ? ) >>>>> This will be Release Candidate 1. >>>>> >>>>> Note: When doing the Maven release and publish process, Maven would >>>>> ask for the version # and update the POM version automatically. >>>>> It is not clear if I need to update the POM with the correct artifact >>>>> version at this point or whether something else will. >>>>> >>>>> Also, I have tried my best to anticipate what the POM should look >>>>> like, but I would like to have someone look at it. >>>>> Specifically, I not sure if the <scm>, <repositories> and the >>>>> <nexus-staging-maven-plugin> are configured correctly. >>>>> >>>>> 4) Create a release branch and onwards: This is a bit confusing as it >>>>> is Gradle driven. I will need to do the equivalent in Maven. >>>>> >>>>> Do you have someone I could video conference with or sit with to >>>>> finish this process? I could come to your office or whatever :) :) >>>>> >>>>> Lee. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Jun 19, 2019 at 11:09 AM Kenneth Knowles <[email protected]> >>>>> wrote: >>>>> >>>>>> Oh, ha, oops. If you flip the branch a bit there will be a pom.xml >>>>>> and we used maven. We had very poor / nonexistant automation. >>>>>> >>>>>> https://github.com/apache/beam/tree/release-2.1.0 (no gradle) >>>>>> https://github.com/apache/beam/tree/release-2.5.0 (gradle is there >>>>>> but we were using the poms) >>>>>> >>>>>> And here's a release guide from back then: >>>>>> https://github.com/apache/beam/blob/e969aefa857aa07156fb9606596db9e7234e54c4/website/src/contribute/release-guide.md >>>>>> >>>>>> On Wed, Jun 19, 2019 at 9:56 AM leerho <[email protected]> wrote: >>>>>> >>>>>>> Kenn, >>>>>>> >>>>>>> This is very helpful as it at least outlines the steps via the >>>>>>> scripts. Unfortunately, we don't use Gradle, we use Maven. I wonder if >>>>>>> you know of a project that has done a similar process description & >>>>>>> scripts >>>>>>> using Maven? >>>>>>> >>>>>>> At least this will give me something to study :) :) >>>>>>> >>>>>>> Thanks! >>>>>>> Lee. >>>>>>> >>>>>>> On Tue, Jun 18, 2019 at 8:19 PM Kenneth Knowles <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Yes, definitely I would love to help. I am a bit time constrained >>>>>>>> right now as my partner is traveling for work and we are closing on a >>>>>>>> new >>>>>>>> home and moving this week and next... >>>>>>>> >>>>>>>> TBH a lot of it is the same as >>>>>>>> https://beam.apache.org/contribute/release-guide/ >>>>>>>> >>>>>>>> And a lot of that is in scripts in >>>>>>>> https://github.com/apache/beam/tree/master/release/src/main/scripts >>>>>>>> >>>>>>>> 1. Set up signing keys >>>>>>>> 2. Build a release candidate (up to you how to do the build) - >>>>>>>> check it in to the right SVN place and stage it in sonatype; I imagine >>>>>>>> you've done that bit before >>>>>>>> 3. Start a [VOTE] thread >>>>>>>> 4. Input the release into reporter.apache.org is helpful; not sure >>>>>>>> if this applies to podlings >>>>>>>> >>>>>>>> Kenn >>>>>>>> >>>>>>>> On Tue, Jun 18, 2019 at 2:52 PM leerho <[email protected]> wrote: >>>>>>>> >>>>>>>>> Kenn, >>>>>>>>> >>>>>>>>> I am stuck. >>>>>>>>> >>>>>>>>> I have completed (I think) the modifications to >>>>>>>>> incubator-datasketches-memory so that it can be released. Before I >>>>>>>>> can >>>>>>>>> move ahead and do the mods on the other repos I need to have the >>>>>>>>> memory >>>>>>>>> repo available as an artifact. >>>>>>>>> >>>>>>>>> So I guess I have to go through the release process. I have tried >>>>>>>>> to read through the documentation, but it is really terrible! It >>>>>>>>> must >>>>>>>>> have been written by lawyers as it is very very verbose and difficult >>>>>>>>> to >>>>>>>>> read with the key information scattered all over the place. >>>>>>>>> >>>>>>>>> What I could use is someone that can walk me through the process, >>>>>>>>> step by step. Once I have gone through that, I'm sure I can >>>>>>>>> reproduce it >>>>>>>>> for all the other repos. >>>>>>>>> >>>>>>>>> Is this something that you can help with? >>>>>>>>> >>>>>>>>> BTW, I contacted INFRA and they don't do this. >>>>>>>>> I also have sent email to [email protected] and have not heard >>>>>>>>> back. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> >>>>>>>>> Lee. >>>>>>>>> >>>>>>>>
