[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645463#comment-16645463 ] ASF GitHub Bot commented on METRON-1769: Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/1188 > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644947#comment-16644947 ] ASF GitHub Bot commented on METRON-1769: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 Yes. I expected we would just do a follow-on when we have cycles. Take the win! :) > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644943#comment-16644943 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @nickwallen @mmiklavc @ottobackwards While I'm loath to look gift +1's in the mouth, the KEYS file issue still isn't resolved. Is everyone okay with me making a follow-on ticket to address it and alter the script as necessary? Possibly add a note in the docs that the jira needs to be resolved (or manually handled) in the release manager switches case? > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643590#comment-16643590 ] ASF GitHub Bot commented on METRON-1769: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1188 +1 from me, great improvements. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643574#comment-16643574 ] ASF GitHub Bot commented on METRON-1769: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1188 +1 by inspection pending your README for @ottobackwards > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643438#comment-16643438 ] ASF GitHub Bot commented on METRON-1769: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 Don't worry about it. I think this is a big step forward. +1 by inspection from me. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643436#comment-16643436 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 I'm open to suggestions on that (is there a way to just sign without a key or something in gpg?). I'll dig around and see if I can find something, too. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643434#comment-16643434 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @nickwallen Not really? The key doesn't need to be in the KEYS file, but the artifacts themselves all have to be signed. We could make it optional but then half of the process isn't being done > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643428#comment-16643428 ] ASF GitHub Bot commented on METRON-1769: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 Is there any way to do a dry run on this myself without having a signing key? What do I enter here? ``` signing key id in 8-byte format (e.g. BADDCAFEDEADBEEF): ``` > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643408#comment-16643408 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @ottobackwards README added, let me know if there's anything else you want to see in it. I limited it to just this script, so the validate and rc check scripts aren't in the README right now. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643213#comment-16643213 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 I assumed this would mostly be documented in the release process wiki, but I can definitely add a readme > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643169#comment-16643169 ] ASF GitHub Bot commented on METRON-1769: Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1188 It may be time for a README for these scripts > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642324#comment-16642324 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @nickwallen Updated the script. I did a few things in the latest commits. * reorganized slightly the order of prompts * Fixed the last (non-KEYS file) TODO * Uncommented out the pushes. * Allow for arbitrary hash or branch name for the base of the RC. * Fixed how the metron-bro-kafka-plugin handles the tag (turns out I tagged it as 0.2.0-release instead of 0.2-release, so I'm just going forward with that). To summarize, at this point the script * Can be run independently for either the top level module or the bro submodule. * The script prompts you for the appropriate inputs to set this up. * Practice run mode allows for the script to be dry-run and outputs the commands it would have run. * The script will let you create an RC from a given revision (or branch), but will not let you cherry-pick commits. Outstanding issues * Adjust the -rcN. Actually not sure what to do about this. I think we'd have to set the base folder name of the tar appropriately at creation time, but the catch is that we sign everything so it's set in stone when we do. We could just not have it on the base folder of the RC tar file and keep the tar file named with -rcN. I don't know if it actually matters * The whole KEYS file thing. From what I can tell this only needs to exist at the root of dist. It might be worthwhile to just keep it in the metron project, and have the instructions for merging that PR tell how to do it separately (e.g. somewhat similar to the site being more standalone). I'm going to reply to the thread about original split to see if anybody has an opinion > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634441#comment-16634441 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @nickwallen Same as my last comment. It's been pretty refactored since the release (in order to handle the discussion we had around releases going forward). I just haven't had time to sit down and finish polishing it, and since the release went out it fell in personal priority. I need to * Fix one TODO in the code * Uncomment the pushes, since I didn't want to risk that. Future fixes * Handle releases not on master (e.g. from a given commit or from a a given branch/commit). We could potentially allow for cherry-picking, but I think the basic cases are sufficient. * Adjust the -rcN. It's probably easy enough to do in here, but I also don't consider it necessary. Outstanding questions * KEYS file for bro plugin repo, as noted above. Feel free to rearrange those if you feel differently, I don't really care if it is follow on, or we do it now. There's probably some more work to handle any hotfix releases, but we've never done one, so I'm not super concerned for right now. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634373#comment-16634373 ] ASF GitHub Bot commented on METRON-1769: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 What is the state of this @justinleet ? Did it work well for you during the last release? > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613917#comment-16613917 ] ASF GitHub Bot commented on METRON-1769: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 > @justinleet: that's because we tar up the rc and just rename the tar.gz file when it gets promoted. It's expected (and occurs on our previous releases). To add onto that, I don't know what other projects actually do about that (they may not, in which case we might want to modify what we do). I can't think of projects that keep the `-rcN`. Just to be sure I checked HBase, Phoenix, and Nifi and none of them seem to do it. We can fix it later though. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613824#comment-16613824 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 To add onto that, I don't know what other projects actually do about that (they may not, in which case we might want to modify what we do). E.g. create the folder without the suffix and rename the resulting tar.gz to have the suffix. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613817#comment-16613817 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 @nickwallen that's because we tar up the rc and just rename the tar.gz file when it gets promoted. It's expected (and occurs on our previous releases) > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613730#comment-16613730 ] ASF GitHub Bot commented on METRON-1769: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 @justinleet One thing I just noticed is that when you download the release tarball from a mirror the file `apache-metron-0.6.0.tar.gz` contains a directory called `apache-metron-0.6.0-rc1`. I would expect it to just be just `apache-metron-0.6.0`. I am not sure if this is something you have scripted here or if it is a manual step in the release process. Either way, not a big deal, just thought I would mention it. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612731#comment-16612731 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 Most of the refactoring for separating the repos is done. The script changed around a lot, but is largely relocating + making input easier/more valid. Couple bit and pieces here and there left to do, but it will currently produce separate repos which artifacts as expected, but not push them. Made some changes to make life easier. - It's more like some of the other scripts, in that it prompts you for info rather than requiring a bunch of flags. Might be worth putting the flags back in case we want to script this into Maven or something? I thought it made it a bit clearer what was happening during testing - More input validation. E.g. version is x.y.z., signing key is reasonably constructed, etc. Outstanding questions - What do we do about the KEYS file? Right now it lives entirely in the Metron repo, It'd be a rare issue, but things could be weird under the following circumstances. 1. There's a new release manager without their key in the KEYS file 2. We're releasing the Bro plugin before Metron. Should this repo have it's own KEYS file? Pull it from the Metron repo? Ignore/document the problem since it's presumably rare? ? Misc - The pushes are currently commented out. Yes, there's practice run mode, but I was not going to even slightly depend on that during a refactoring while a release was going on. - Nope, this isn't handling non-master branches. Probably pretty easy to do, just didn't. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609240#comment-16609240 ] ASF GitHub Bot commented on METRON-1769: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 FWIW - Your changes to `metron-rc-check` worked perfectly. I validated that during 0.6.0 RC1 > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609236#comment-16609236 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 Per [a discuss thread](https://lists.apache.org/thread.html/b651737261c712deb83b11750033372916698780c35522263e949095@%3Cdev.metron.apache.org%3E), going forward we plan on splitting the releases entirely. Given that plan, I'm going to refactor this to produce a single RC at a time. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607691#comment-16607691 ] ASF GitHub Bot commented on METRON-1769: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 > I'm assuming this always pulls HEAD from master to cut the release. Do we need or desire any support for cutting a release from a non-HEAD commit? I'd also completely support doing this as a follow-on, if you think that is best @justinleet. It is certainly not necessary for this PR. And what you've done has taken a manual process and automated it, which is great. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607656#comment-16607656 ] ASF GitHub Bot commented on METRON-1769: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1188 Trying this again on the PR itself :-) Yeah, the Angular upgrade was the other bit that came to mind. Shane's PR for the Angular upgrade has the necessary +1's, but @nickwallen you had requested we hold off on that for this release (which I completely agree with). https://github.com/apache/metron/pull/1096 > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607465#comment-16607465 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 I could be wrong, but I think we've been fairly loose about it. Matt may have done some management of it, but it would have been entirely manual. In most cases we should be able to release from master without concern (after all, if it can't be released, why is it in master?) Maintenance releases are definitely something that should be considered (although I don't think we've ever done one). I believe that if we were doing a maintenance release, we'd just be keeping a branch up to date and releasing from that branch. Per the release process wiki page > Maintaining a Maintenance Branch > >Being able to maintain the previous release train, with only critical or important bug fixes and security fixes (generally not new features) for users who are averse to frequent large changes is very important for production use. They get stability, while the new feature release code line proceeds as fast as the community wishes. > > When needed, and with community discussion, create a Maintenance Branch for the previous Metron release by incrementing the *third* digit of the previous release like so 0.[FR].[*MR++*]. All patches to the previous Metron release will be checked in under the MR branch and where it makes sense also under the master branch. All new features will be checked in under the master branch. Maybe we can address at least the majority of cases for both of these by adding an option for a commit/branch to use as the basis of the RC? To be more thorough, we'd probably need to be more flexible on what exactly goes in, but that seems like a fairly straightforward way to build on what we have. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607320#comment-16607320 ] ASF GitHub Bot commented on METRON-1769: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1188 > I'm assuming this always pulls HEAD from master to cut the release. Do we need or desire any support for cutting a release from a non-HEAD commit? It would be very useful to continue to merge PRs into master while a release is being voted on. I had thought that @mattf-horton use to do the releases in such a way that this was not a problem, but I could be wrong. For example, this morning I merged PR #1174 into master that I don't necessarily want in the next release. I didn't think about the potential impact to the release if we have to cut a new RC. Sorry about that @justinleet . > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607257#comment-16607257 ] ASF GitHub Bot commented on METRON-1769: Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/1188 I'm assuming this always pulls HEAD from master to cut the release. Do we need or desire any support for cutting a release from a non-HEAD commit? Also, wondering if we need to consider maintenance releases at all where we might cherry-pick commits from master or have a feature branch specifically for the maint release. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16606149#comment-16606149 ] ASF GitHub Bot commented on METRON-1769: Github user justinleet commented on the issue: https://github.com/apache/metron/pull/1188 That's definitely an option. I know offhand there's a few more similar flags (e.g. -o pipefail), which may also be useful. It might be sufficient, at least as a first cut. > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16606125#comment-16606125 ] ASF GitHub Bot commented on METRON-1769: Github user JonZeolla commented on the issue: https://github.com/apache/metron/pull/1188 For error handling, we should look into `set -e` and the related downfalls. It has been a while since I looked into it. http://mywiki.wooledge.org/BashFAQ/105 > Script creation of a release candidate > -- > > Key: METRON-1769 > URL: https://issues.apache.org/jira/browse/METRON-1769 > Project: Metron > Issue Type: Improvement >Reporter: Justin Leet >Assignee: Justin Leet >Priority: Major > > Our process at: > https://cwiki.apache.org/confluence/display/METRON/Release+Process is fairly > error prone. There's a lot of typing out versions and other tedious busywork. > In particular, this JIRA is for steps 4/5, as they're the most egregious. > This script should be able to produce both the core artifacts as well as the > bro kafka plugin artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (METRON-1769) Script creation of a release candidate
[ https://issues.apache.org/jira/browse/METRON-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16605927#comment-16605927 ] ASF GitHub Bot commented on METRON-1769: GitHub user justinleet opened a pull request: https://github.com/apache/metron/pull/1188 METRON-1769: Script creation of a release candidate ## Contributor Comments Preliminary script that I wanted to make available for feedback. It basically scripts out steps 4 and 5 of the RC creation process https://cwiki.apache.org/confluence/display/METRON/Release+Process Basically it will * Checkout the various repos needed during the release process * Setup branches/directories as needed * Create the release artifacts for both core metron as well as metron-bro-plugin-kafka * This includes the tar.gz * The signatures (SHA-256 and SHA-512. MD5 and SHA-1 files are no longer included in the releases) * The various supplementary files (LICENSE, CHANGES, etc). It also has a practice run mode that won't push anything to the servers. Things that need to be improved * Need to handle only releasing core and not the plugin. Probably the other way around, too? * Error handling when things go wrong (e.g. couldn't sign artifacts). Right now it's standard Bash "Keep going until the script crashes" * Input validation around version format and such? * Might be nice to tie this into maven Sidenote, the general process around the plugin is really rough and undocumented right now. We should clean that up (and preferably make it match formatting wise with the rest of Metron. This might be things like version numbers as discussed on release threads, but even the tag naming conventions differ, along with minor things like '-' vs '_' which made doing this extra fun) ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [ ] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh ``` - [ ] Have you written or updated unit tests and or integration tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/justinleet/metron release_utils Alternatively you can review and apply these changes as the patch at: https://github.com/apache/metron/pull/1188.patch To close this pull request, make a
