[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 Yes. I expected we would just do a follow-on when we have cycles. Take the win! :) ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @nickwallen @mmiklavc @ottobackwards While I'm loath to look gift +1's in the mouth, the KEYS file issue still isn't resolved. Is everyone okay with me making a follow-on ticket to address it and alter the script as necessary? Possibly add a note in the docs that the jira needs to be resolved (or manually handled) in the release manager switches case? ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1188 +1 from me, great improvements. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1188 +1 by inspection pending your README for @ottobackwards ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 Don't worry about it. I think this is a big step forward. +1 by inspection from me. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 I'm open to suggestions on that (is there a way to just sign without a key or something in gpg?). I'll dig around and see if I can find something, too. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @nickwallen Not really? The key doesn't need to be in the KEYS file, but the artifacts themselves all have to be signed. We could make it optional but then half of the process isn't being done ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 Is there any way to do a dry run on this myself without having a signing key? What do I enter here? ``` signing key id in 8-byte format (e.g. BADDCAFEDEADBEEF): ``` ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @ottobackwards README added, let me know if there's anything else you want to see in it. I limited it to just this script, so the validate and rc check scripts aren't in the README right now. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 I assumed this would mostly be documented in the release process wiki, but I can definitely add a readme ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1188 It may be time for a README for these scripts ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @nickwallen Updated the script. I did a few things in the latest commits. * reorganized slightly the order of prompts * Fixed the last (non-KEYS file) TODO * Uncommented out the pushes. * Allow for arbitrary hash or branch name for the base of the RC. * Fixed how the metron-bro-kafka-plugin handles the tag (turns out I tagged it as 0.2.0-release instead of 0.2-release, so I'm just going forward with that). To summarize, at this point the script * Can be run independently for either the top level module or the bro submodule. * The script prompts you for the appropriate inputs to set this up. * Practice run mode allows for the script to be dry-run and outputs the commands it would have run. * The script will let you create an RC from a given revision (or branch), but will not let you cherry-pick commits. Outstanding issues * Adjust the -rcN. Actually not sure what to do about this. I think we'd have to set the base folder name of the tar appropriately at creation time, but the catch is that we sign everything so it's set in stone when we do. We could just not have it on the base folder of the RC tar file and keep the tar file named with -rcN. I don't know if it actually matters * The whole KEYS file thing. From what I can tell this only needs to exist at the root of dist. It might be worthwhile to just keep it in the metron project, and have the instructions for merging that PR tell how to do it separately (e.g. somewhat similar to the site being more standalone). I'm going to reply to the thread about original split to see if anybody has an opinion ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @nickwallen Same as my last comment. It's been pretty refactored since the release (in order to handle the discussion we had around releases going forward). I just haven't had time to sit down and finish polishing it, and since the release went out it fell in personal priority. I need to * Fix one TODO in the code * Uncomment the pushes, since I didn't want to risk that. Future fixes * Handle releases not on master (e.g. from a given commit or from a a given branch/commit). We could potentially allow for cherry-picking, but I think the basic cases are sufficient. * Adjust the -rcN. It's probably easy enough to do in here, but I also don't consider it necessary. Outstanding questions * KEYS file for bro plugin repo, as noted above. Feel free to rearrange those if you feel differently, I don't really care if it is follow on, or we do it now. There's probably some more work to handle any hotfix releases, but we've never done one, so I'm not super concerned for right now. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 What is the state of this @justinleet ? Did it work well for you during the last release? ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 > @justinleet: that's because we tar up the rc and just rename the tar.gz file when it gets promoted. It's expected (and occurs on our previous releases). To add onto that, I don't know what other projects actually do about that (they may not, in which case we might want to modify what we do). I can't think of projects that keep the `-rcN`. Just to be sure I checked HBase, Phoenix, and Nifi and none of them seem to do it. We can fix it later though. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 To add onto that, I don't know what other projects actually do about that (they may not, in which case we might want to modify what we do). E.g. create the folder without the suffix and rename the resulting tar.gz to have the suffix. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @nickwallen that's because we tar up the rc and just rename the tar.gz file when it gets promoted. It's expected (and occurs on our previous releases) ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 @justinleet One thing I just noticed is that when you download the release tarball from a mirror the file `apache-metron-0.6.0.tar.gz` contains a directory called `apache-metron-0.6.0-rc1`. I would expect it to just be just `apache-metron-0.6.0`. I am not sure if this is something you have scripted here or if it is a manual step in the release process. Either way, not a big deal, just thought I would mention it. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 Most of the refactoring for separating the repos is done. The script changed around a lot, but is largely relocating + making input easier/more valid. Couple bit and pieces here and there left to do, but it will currently produce separate repos which artifacts as expected, but not push them. Made some changes to make life easier. - It's more like some of the other scripts, in that it prompts you for info rather than requiring a bunch of flags. Might be worth putting the flags back in case we want to script this into Maven or something? I thought it made it a bit clearer what was happening during testing - More input validation. E.g. version is x.y.z., signing key is reasonably constructed, etc. Outstanding questions - What do we do about the KEYS file? Right now it lives entirely in the Metron repo, It'd be a rare issue, but things could be weird under the following circumstances. 1. There's a new release manager without their key in the KEYS file 2. We're releasing the Bro plugin before Metron. Should this repo have it's own KEYS file? Pull it from the Metron repo? Ignore/document the problem since it's presumably rare? ? Misc - The pushes are currently commented out. Yes, there's practice run mode, but I was not going to even slightly depend on that during a refactoring while a release was going on. - Nope, this isn't handling non-master branches. Probably pretty easy to do, just didn't. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 FWIW - Your changes to `metron-rc-check` worked perfectly. I validated that during 0.6.0 RC1 ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 Per [a discuss thread](https://lists.apache.org/thread.html/b651737261c712deb83b11750033372916698780c35522263e949095@%3Cdev.metron.apache.org%3E), going forward we plan on splitting the releases entirely. Given that plan, I'm going to refactor this to produce a single RC at a time. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 > I'm assuming this always pulls HEAD from master to cut the release. Do we need or desire any support for cutting a release from a non-HEAD commit? I'd also completely support doing this as a follow-on, if you think that is best @justinleet. It is certainly not necessary for this PR. And what you've done has taken a manual process and automated it, which is great. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1188 Trying this again on the PR itself :-) Yeah, the Angular upgrade was the other bit that came to mind. Shane's PR for the Angular upgrade has the necessary +1's, but @nickwallen you had requested we hold off on that for this release (which I completely agree with). https://github.com/apache/metron/pull/1096 ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 I could be wrong, but I think we've been fairly loose about it. Matt may have done some management of it, but it would have been entirely manual. In most cases we should be able to release from master without concern (after all, if it can't be released, why is it in master?) Maintenance releases are definitely something that should be considered (although I don't think we've ever done one). I believe that if we were doing a maintenance release, we'd just be keeping a branch up to date and releasing from that branch. Per the release process wiki page > Maintaining a Maintenance Branch > >Being able to maintain the previous release train, with only critical or important bug fixes and security fixes (generally not new features) for users who are averse to frequent large changes is very important for production use. They get stability, while the new feature release code line proceeds as fast as the community wishes. > > When needed, and with community discussion, create a Maintenance Branch for the previous Metron release by incrementing the *third* digit of the previous release like so 0.[FR].[*MR++*]. All patches to the previous Metron release will be checked in under the MR branch and where it makes sense also under the master branch. All new features will be checked in under the master branch. Maybe we can address at least the majority of cases for both of these by adding an option for a commit/branch to use as the basis of the RC? To be more thorough, we'd probably need to be more flexible on what exactly goes in, but that seems like a fairly straightforward way to build on what we have. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 > I'm assuming this always pulls HEAD from master to cut the release. Do we need or desire any support for cutting a release from a non-HEAD commit? It would be very useful to continue to merge PRs into master while a release is being voted on. I had thought that @mattf-horton use to do the releases in such a way that this was not a problem, but I could be wrong. For example, this morning I merged PR #1174 into master that I don't necessarily want in the next release. I didn't think about the potential impact to the release if we have to cut a new RC. Sorry about that @justinleet . ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1188 I'm assuming this always pulls HEAD from master to cut the release. Do we need or desire any support for cutting a release from a non-HEAD commit? Also, wondering if we need to consider maintenance releases at all where we might cherry-pick commits from master or have a feature branch specifically for the maint release. ---
[GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 That's definitely an option. I know offhand there's a few more similar flags (e.g. -o pipefail), which may also be useful. It might be sufficient, at least as a first cut. ---
