Sorry, missed one. I think this is the one Jon prefers. (5) The other alternative is just to complete Jon's roadmap BEFORE the next release.
On Thu, Nov 16, 2017 at 10:37 AM, Nick Allen <n...@nickallen.org> wrote: > Just to layout some other alternatives... > > (1) [Jon] Make a release in apache/metron-bro-plugin-kafka, then update > Ansible to pin to that release. > > (2) [Jon] Update Ansible to pin to a specific commit. > > (3) Wouldn't the submodule approach solve this? I imagine that if we use > a submodule, then all of the Bro plugin code will be contained directly in > each Metron release. We would just need to change that small bit of > Ansible code so that it uses the code directly (like before) instead of > going to Git and checking it out. > > (4) Revert PR #837 because as you pointed out this approach does not work > well with releases. That would give us enough time to come up with a > sensible approach and do that. > > > > > > > On Thu, Nov 16, 2017 at 10:21 AM, zeo...@gmail.com <zeo...@gmail.com> > wrote: > >> The problem is that we're currently pinning to master and if something in >> the plugin breaks backward compatibility, our prior releases will be >> perpetually broken, which is why I suggest it blocks the upcoming release. >> >> The alternative would be to update ansible to pin to a specific commit or >> to make a release in apache/metron-bro-plugin-kafka sooner rather than >> later and pin to its branch/tag. That feels like a waste of time though, >> as the 2.5.2 upgrade is somewhat trivial. >> >> Jon >> >> On Thu, Nov 16, 2017 at 10:14 AM Nick Allen <n...@nickallen.org> wrote: >> >> > While I think that the Bro work is super valuable, Jon, I am not sure >> that >> > we need to block the next release for it. In my mind, the "theme" of >> the >> > next release is Metaalerts running on ES 2.x. I'd prefer to just stick >> > with that. >> > >> > I was hoping we could get the remaining "Metaalerts + ES 2.x" PRs merged >> > and start cutting release candidates ASAP. That could be possible by >> end >> > of week based on the "big one" (METRON-1289) getting merged in last >> night. >> > >> > Of course, if you happen to get all the Bro work done in time, then >> > definitely let's include it in the next release. But I am wary of >> blocking >> > the release for that work. No need for you to rush through it. >> > >> > Just one man's opinion. Would like to hear feedback from more of the >> > community. >> > >> > >> > >> > On Thu, Nov 16, 2017 at 8:01 AM, zeo...@gmail.com <zeo...@gmail.com> >> > wrote: >> > >> > > The way master's full-dev is set up right now is non optimal for the >> bro >> > > plugin configuration, and I would like to complete the roadmap I >> outlined >> > > in my discuss thread before a release. I have a PR open against >> > > Apache/metron-bro-plugin-kafka right now to turn it into a plugin, >> and I >> > > expect it will take me until end of next week at the latest to have >> the >> > > rest of the work done to move us to 2.5.2, and to pin to a specific >> > package >> > > version. At that point I'm comfortable with a release. >> > > >> > > Jon >> > > >> > > On Wed, Nov 15, 2017, 18:57 Matt Foley <ma...@apache.org> wrote: >> > > >> > > > I’ve been listening. Looks like there are still a number of major >> > issues >> > > > to be committed first, right? >> > > > The discussion on this thread constitutes sufficient engagement, I >> > think, >> > > > especially given the Subject line :-) >> > > > Would the folks working on the 6 issues listed by Nick care to >> suggest >> > a >> > > > cut-off date by which they’ll probably have those fixes in? >> > > > I’ll be happy to run the release process, if the community so >> wishes, >> > as >> > > > soon as those issues are committed. >> > > > >> > > > I guess I should, pro forma, send the list of commits already in >> since >> > > the >> > > > last release. I’ll do that today. >> > > > Also, if anyone wishes to raise a hand and propose additional >> commits >> > are >> > > > needed, please do so on this thread. >> > > > Thanks, >> > > > --Matt >> > > > >> > > > >> > > > On 11/15/17, 2:09 PM, "Casey Stella" <ceste...@gmail.com> wrote: >> > > > >> > > > I'd say that if a release is this imminent that we had better >> > notify >> > > > the >> > > > release manager who will make a release announcement, Nick. >> Matt, >> > > are >> > > > you >> > > > tuning in to this? >> > > > >> > > > On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen < >> n...@nickallen.org> >> > > > wrote: >> > > > >> > > > > Hi Guys - >> > > > > >> > > > > I want to follow-up on this discussion. It sounds like most >> > people >> > > > are in >> > > > > agreement with the general approach. >> > > > > >> > > > > A lot of people have been working hard on Metaalerts and >> > > > Elasticsearch. I >> > > > > have checked-in with those doing the heavy lifting and have >> > > compiled >> > > > a more >> > > > > detailed plan based on where we are at now. To the best of my >> > > > knowledge >> > > > > here is the plan of attack for finishing out this effort. >> > > > > >> > > > > (1) First, METRON-1289 needs to go in. This one was a >> fairly >> > big >> > > > effort >> > > > > and I am hearing that we are pretty close. >> > > > > >> > > > > (2) METRON-1294 fixes an issue in how field types are >> > looked-up. >> > > > > >> > > > > (3) METRON-1290 is next. While this may have been fixed in >> > > > M-1289, there >> > > > > may be some test cases we want from this PR. >> > > > > >> > > > > (4) METRON-1301 addresses a problem with the sorting logic. >> > > > > >> > > > > (5) METRON-1291 fixes an issue with escalation of >> metaalerts. >> > > > > >> > > > > (6) That leads us to Raghu's UI work in METRON-1252. This >> > > > introduces the >> > > > > UI bits that depend on all the previous backend work. >> > > > > >> > > > > (7) At this point, we should have our best effort at running >> > > > Metaalerts >> > > > > on Elasticsearch 2.x. I propose that we cut a release here. >> > > > > >> > > > > (8) After we cut the release, we can introduce the work for >> ES >> > > 5.x >> > > > in >> > > > > METRON-939. I know we will need lots of help testing and >> > reviewing >> > > > this >> > > > > one. >> > > > > >> > > > > Please correct me if I am wrong. I will try and send out >> updates >> > > as >> > > > we >> > > > > make progress. >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > On Mon, Nov 6, 2017 at 1:03 PM, zeo...@gmail.com < >> > zeo...@gmail.com >> > > > >> > > > wrote: >> > > > > >> > > > > > I agree, I think it's very reasonable to move in line with >> > Nick's >> > > > > > proposal. I would also suggest that we outline what the >> target >> > > > versions >> > > > > > would be to add in the METRON-777 components, since it has >> been >> > > > > functional >> > > > > > for a very long time but not reviewed and has some really >> > > rockstar >> > > > > > improvements. >> > > > > > >> > > > > > Jon >> > > > > > >> > > > > > On Mon, Nov 6, 2017 at 12:56 PM Otto Fowler < >> > > > ottobackwa...@gmail.com> >> > > > > > wrote: >> > > > > > >> > > > > > > I think the ES cutover should be the start of the 0.5.x >> > series, >> > > > and we >> > > > > > > continue on with 0.4.x for the >> > > > > > > metadata improvements etc. We could chose to focus >> 0.5.x’s >> > > first >> > > > > > releases >> > > > > > > on not only ES but >> > > > > > > getting a handle on kibana and the mpack situation as >> well. >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > On November 6, 2017 at 12:48:45, Michael Miklavcic ( >> > > > > > > michael.miklav...@gmail.com) wrote: >> > > > > > > >> > > > > > > I agree with your proposal, Nick. I think having a >> > stabilizing >> > > > release >> > > > > > > prior to upgrading ES/Kibana makes sense. >> > > > > > > >> > > > > > > On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen < >> > n...@nickallen.org >> > > > >> > > > wrote: >> > > > > > > >> > > > > > > > I would like to start a discussion around upcoming >> > releases. >> > > > We have >> > > > > a >> > > > > > > > couple separate significant tracks of work that we need >> to >> > > > reconcile >> > > > > in >> > > > > > > our >> > > > > > > > release schedule. >> > > > > > > > >> > > > > > > > (1) We have had (and have in review) a good number of >> bug >> > > fixes >> > > > > > required >> > > > > > > to >> > > > > > > > support Metaalerts on the existing Elasticsearch 2.x >> > > > infrastructure. >> > > > > > > > >> > > > > > > > >> > > > > > > > (2) We also have ongoing work to upgrade our >> infrastructure >> > > to >> > > > > > > > Elasticsearch 5.x, which will not be backwards >> compatible. >> > > > > > > > >> > > > > > > > >> > > > > > > > I would like to see a release that has our best work on >> ES >> > > 2.x >> > > > before >> > > > > > we >> > > > > > > > migrate to 5.x. I would propose the following. >> > > > > > > > >> > > > > > > > Release N+1: Introduce Metaalerts running on ES 2.x >> > > > > > > > >> > > > > > > > Release N+2: Cut-over to ES 5.x >> > > > > > > > >> > > > > > > > >> > > > > > > > (Q) Is it worth cutting a separate release for ES 2.x? >> Is >> > > > there a >> > > > > > better >> > > > > > > > way to handle the cut-over to 5.x? >> > > > > > > > >> > > > > > > >> > > > > > -- >> > > > > > >> > > > > > Jon >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> > > > -- >> > > >> > > Jon >> > > >> > >> -- >> >> Jon >> > >
