Based on the latest comments, it looks to me like this feature for the UI is nearly complete at this point. I independently spoke with Tibor Meller offline about this and am told that 8 days were spent on testing, which is how METRON-2034 was found in the first place. That PR needs some cleanup because quite a few commits don't appear related to it at all, but it seems like this is pretty well tested, and I think we should probably get this into the release. Had this simply been a feature gap, I'd be inclined to keep my +1 as is. Full dev is directed at developers, so I'm even more inclined to be just as happy with a README notice about something like this in the future. And as pointed out, we've already had one release with this state of affairs. However, since this is on the verge of being done, I am in agreement with Otto and change my vote to a binding -1.
As a side note, while I haven't reviewed all the discussion about the aggregation full dev decision to note if this was done or not, it's worth noting (or re-emphasizing) that we should probably take the extra step of pointing out breaking changes on the dev list, whether or not they were discussed on individual PRs. I expect that committers will always have a higher "have read" ratio on dev list emails than on individual PR's and their respective comments lists. This ensures decisions like this are always broadcast for the largest committer audience. We specifically landed on this approach with the dev docs for the writer infrastructure as well as the intermittent test failures, and we were able to address them both up as planned. Best, Mike On Mon, Apr 29, 2019 at 1:36 PM Ryan Merriman <merrim...@gmail.com> wrote: > I am working on the backend change mentioned above ( > https://issues.apache.org/jira/browse/METRON-2034) and should a PR up > today. > > On Mon, Apr 29, 2019 at 1:16 PM Tamás Fodor <ftamas.m...@gmail.com> wrote: > > > As Justin pointed out, we've already implemented the frontend related > part > > of the aggregation in https://github.com/apache/metron/pull/1360. Since > > it's a very big changeset, we would like to double check it again before > we > > merge it back to master. Also, we're waiting for a small change in the > > backend code to fully cover everything related to parser aggregation. > Once > > we have introduced this small patch and fully tested it manually we can > > solve the issue with the aggregated sensors on the management UI. > > > > On Sun, Apr 28, 2019 at 8:30 PM Nick Allen <n...@nickallen.org> wrote: > > > > > I agree with Justin. My +1 stands. > > > > > > Considering that this is a known gap, we have already released with > this > > > gap, and we have a backlog of numerous improvements that should be > > released > > > to the community, I am not in favor of delaying the release. Metron > > > provides a wide variety of functionality at varying levels of maturity. > > > This is to be expected. If we expect perfection, we will never get a > > > release out. > > > > > > > > > On Sat, Apr 27, 2019 at 6:12 PM Justin Leet <justinjl...@gmail.com> > > wrote: > > > > > > > Mike is correct, that is because of the combination of full dev > > > > restrictions and the lack of support in the configuration UI for > parser > > > > aggregation. This was introduced in > > > > https://github.com/apache/metron/pull/1207 and also was true of the > > last > > > > release. Currently, parser aggregation is an advanced/manual feature > > > whose > > > > (bare minimum) configuration can be done via Ambari, out of > > convenience. > > > > > > > > I haven't looked into it, but > > https://github.com/apache/metron/pull/1360 > > > > is > > > > likely the work for this (and need additional work before merging). > > > > > > > > I'm personally letting my binding +1 stand, although I would support > > > either > > > > ensuring we get that PR cleaned up and in and/or additional > > documentation > > > > regarding the current limitations of this feature. > > > > > > > > > > > > On Sat, Apr 27, 2019 at 2:38 PM Anand Subramanian < > > > > asubraman...@hortonworks.com> wrote: > > > > > > > > > I can confirm that I've seen the Mgmt UI shows the sensor status > > > > correctly > > > > > when they run as single topologies. > > > > > > > > > > -Anand > > > > > > > > > > On 4/27/19, 11:37 PM, "Michael Miklavcic" < > > > michael.miklav...@gmail.com> > > > > > wrote: > > > > > > > > > > I believe that is bc of parser aggregation. The UI does not > > support > > > > it > > > > > currently. IIRC there was a PR to change the bro, snort, and > yaf > > > > > sensors to > > > > > aggregated bc full dev didn't have enough resources. The upshot > > is > > > > > that the > > > > > UI still works for single sensors, but the feature for enabling > > > > > aggregated > > > > > sensors has not yet been completed. > > > > > > > > > > On Sat, Apr 27, 2019, 11:33 AM Otto Fowler < > > > ottobackwa...@gmail.com> > > > > > wrote: > > > > > > > > > > > -1 > > > > > > > > > > > > Ran the script and ran full dev, all good. > > > > > > In the configuration ui, the status of the sensors is not > > > correct. > > > > > It > > > > > > does not show any running, but they are running in storm and > > the > > > > > data was > > > > > > moved correctly. > > > > > > > > > > > > > > > > > > On April 26, 2019 at 09:58:02, Otto Fowler ( > > > > ottobackwa...@gmail.com) > > > > > > wrote: > > > > > > > > > > > > Curious Anand, > > > > > > are your steps for bringing up an open stack cluster > something > > we > > > > > could > > > > > > script like the AWS stuff? > > > > > > > > > > > > > > > > > > On April 26, 2019 at 09:35:29, Anand Subramanian ( > > > > > > asubraman...@hortonworks.com) wrote: > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > * Built RPMs and mpacks. > > > > > > * Brought up Metron stack on 12-node CentOS 7 openstack > > cluster. > > > > > > * Ran sensor-stubs and validated events in the Alerts UI for > > the > > > > > default > > > > > > sensors. > > > > > > * Management UI, Alerts UI and Swagger UI sanity check > > > > > > > > > > > > Regards, > > > > > > Anand > > > > > > > > > > > > On 4/26/19, 5:18 AM, "Nick Allen" <n...@nickallen.org> > wrote: > > > > > > > > > > > > +1 Verified release with all documented steps and ran up Full > > > Dev. > > > > > > > > > > > > On Thu, Apr 25, 2019 at 6:10 PM Michael Miklavcic < > > > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > > > > > Ok cool, just finished the validation and updated the steps > > in > > > > the > > > > > doc to > > > > > > > reflect the current code base. > > > > > > > > > > > > > > On Thu, Apr 25, 2019 at 3:45 PM Nick Allen < > > n...@nickallen.org > > > > > > > > > wrote: > > > > > > > > > > > > > > > No voting required. Those are just docs. Whoever is > willing > > > to > > > > > correct > > > > > > > > and has access, should be able to. Good catch. > > > > > > > > > > > > > > > > On Thu, Apr 25, 2019 at 4:32 PM Michael Miklavcic < > > > > > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > > > > > > > > > We're also not "incubator-metron" any longer. Do we > > require > > > > > any kind > > > > > > of > > > > > > > > > voting or +1 on that verification page to make > > corrections > > > to > > > > > it? > > > > > > > > > > > > > > > > > > On Thu, Apr 25, 2019 at 2:29 PM Michael Miklavcic < > > > > > > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > fyi, the steps in this doc have changed slightly per > > this > > > > > naming > > > > > > > > > > convention change as well - > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 25, 2019 at 1:25 PM Justin Leet < > > > > > justinjl...@gmail.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > >> For everyone taking the time to validate and vote on > > the > > > > > RC, there > > > > > > > is > > > > > > > > a > > > > > > > > > >> caveat. The naming conventions for the two repos are > > now > > > > > aligned > > > > > > > > > >> (<repo_name>_<version>, instead of being '-' in the > > main > > > > > repo and > > > > > > > '_' > > > > > > > > in > > > > > > > > > >> the plugin repo) along with the location of the KEYS > > > file, > > > > > I have > > > > > > a > > > > > > > PR > > > > > > > > > out > > > > > > > > > >> to update the metron-rc-check script ( > > > > > > > > > >> https://github.com/apache/metron/pull/1394). > > > > > > > > > >> > > > > > > > > > >> This accounts for both of these changes, and should > > > allow > > > > > the > > > > > > script > > > > > > > > to > > > > > > > > > be > > > > > > > > > >> run normally. > > > > > > > > > >> > > > > > > > > > >> On Thu, Apr 25, 2019 at 3:22 PM Justin Leet < > > > > > > justinjl...@gmail.com> > > > > > > > > > >> wrote: > > > > > > > > > >> > > > > > > > > > >> > This is a call to vote on releasing Apache Metron > > > 0.7.1 > > > > > > > > > >> > > > > > > > > > > >> > Full list of changes in this release: > > > > > > > > > >> > > > > > > https://dist.apache.org/repos/dist/dev/metron/0.7.1-RC1/CHANGES > > > > > > > > > >> > The tag to be voted upon is: > > > > > > > > > >> > apache-metron_0.7.1-rc1 > > > > > > > > > >> > > > > > > > > > > >> > The source archives being voted upon can be found > > > here: > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/metron/0.7.1-RC1/apache-metron_0.7.1-rc1.tar.gz > > > > > > > > > >> > > > > > > > > > > >> > Other release files, signatures and digests can be > > > found > > > > > here: > > > > > > > > > >> > > > > > https://dist.apache.org/repos/dist/dev/metron/0.7.1-RC1/ > > > > > > > > > >> > > > > > > > > > > >> > The release artifacts are signed with the > following > > > key: > > > > > > > > > >> > > > > https://dist.apache.org/repos/dist/release/metron/KEYS > > > > > > > > > >> > Please vote on releasing this package as Apache > > Metron > > > > > 0.7.1-RC1 > > > > > > > > > >> > > > > > > > > > > >> > When voting, please list the actions taken to > verify > > > the > > > > > > release. > > > > > > > > > >> > > > > > > > > > > >> > Recommended build validation and verification > > > > > instructions are > > > > > > > > posted > > > > > > > > > >> > here: > > > > > > > > > >> > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds > > > > > > > > > >> > > > > > > > > > > >> > This vote will be open for until 4pm EDT on > Tuesday > > > > April > > > > > 30 > > > > > > 2019, > > > > > > > > to > > > > > > > > > >> > account for the weekend.. > > > > > > > > > >> > > > > > > > > > > >> > [ ] +1 Release this package as Apache Metron > > 0.7.1-RC1 > > > > > > > > > >> > > > > > > > > > > >> > [ ] 0 No opinion > > > > > > > > > >> > > > > > > > > > > >> > [ ] -1 Do not release this package because... > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
