As Justin pointed out, we've already implemented the frontend related part of the aggregation in https://github.com/apache/metron/pull/1360. Since it's a very big changeset, we would like to double check it again before we merge it back to master. Also, we're waiting for a small change in the backend code to fully cover everything related to parser aggregation. Once we have introduced this small patch and fully tested it manually we can solve the issue with the aggregated sensors on the management UI.
On Sun, Apr 28, 2019 at 8:30 PM Nick Allen <n...@nickallen.org> wrote: > I agree with Justin. My +1 stands. > > Considering that this is a known gap, we have already released with this > gap, and we have a backlog of numerous improvements that should be released > to the community, I am not in favor of delaying the release. Metron > provides a wide variety of functionality at varying levels of maturity. > This is to be expected. If we expect perfection, we will never get a > release out. > > > On Sat, Apr 27, 2019 at 6:12 PM Justin Leet <justinjl...@gmail.com> wrote: > > > Mike is correct, that is because of the combination of full dev > > restrictions and the lack of support in the configuration UI for parser > > aggregation. This was introduced in > > https://github.com/apache/metron/pull/1207 and also was true of the last > > release. Currently, parser aggregation is an advanced/manual feature > whose > > (bare minimum) configuration can be done via Ambari, out of convenience. > > > > I haven't looked into it, but https://github.com/apache/metron/pull/1360 > > is > > likely the work for this (and need additional work before merging). > > > > I'm personally letting my binding +1 stand, although I would support > either > > ensuring we get that PR cleaned up and in and/or additional documentation > > regarding the current limitations of this feature. > > > > > > On Sat, Apr 27, 2019 at 2:38 PM Anand Subramanian < > > asubraman...@hortonworks.com> wrote: > > > > > I can confirm that I've seen the Mgmt UI shows the sensor status > > correctly > > > when they run as single topologies. > > > > > > -Anand > > > > > > On 4/27/19, 11:37 PM, "Michael Miklavcic" < > michael.miklav...@gmail.com> > > > wrote: > > > > > > I believe that is bc of parser aggregation. The UI does not support > > it > > > currently. IIRC there was a PR to change the bro, snort, and yaf > > > sensors to > > > aggregated bc full dev didn't have enough resources. The upshot is > > > that the > > > UI still works for single sensors, but the feature for enabling > > > aggregated > > > sensors has not yet been completed. > > > > > > On Sat, Apr 27, 2019, 11:33 AM Otto Fowler < > ottobackwa...@gmail.com> > > > wrote: > > > > > > > -1 > > > > > > > > Ran the script and ran full dev, all good. > > > > In the configuration ui, the status of the sensors is not > correct. > > > It > > > > does not show any running, but they are running in storm and the > > > data was > > > > moved correctly. > > > > > > > > > > > > On April 26, 2019 at 09:58:02, Otto Fowler ( > > ottobackwa...@gmail.com) > > > > wrote: > > > > > > > > Curious Anand, > > > > are your steps for bringing up an open stack cluster something we > > > could > > > > script like the AWS stuff? > > > > > > > > > > > > On April 26, 2019 at 09:35:29, Anand Subramanian ( > > > > asubraman...@hortonworks.com) wrote: > > > > > > > > +1 (non-binding) > > > > > > > > * Built RPMs and mpacks. > > > > * Brought up Metron stack on 12-node CentOS 7 openstack cluster. > > > > * Ran sensor-stubs and validated events in the Alerts UI for the > > > default > > > > sensors. > > > > * Management UI, Alerts UI and Swagger UI sanity check > > > > > > > > Regards, > > > > Anand > > > > > > > > On 4/26/19, 5:18 AM, "Nick Allen" <n...@nickallen.org> wrote: > > > > > > > > +1 Verified release with all documented steps and ran up Full > Dev. > > > > > > > > On Thu, Apr 25, 2019 at 6:10 PM Michael Miklavcic < > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > Ok cool, just finished the validation and updated the steps in > > the > > > doc to > > > > > reflect the current code base. > > > > > > > > > > On Thu, Apr 25, 2019 at 3:45 PM Nick Allen <n...@nickallen.org > > > > > wrote: > > > > > > > > > > > No voting required. Those are just docs. Whoever is willing > to > > > correct > > > > > > and has access, should be able to. Good catch. > > > > > > > > > > > > On Thu, Apr 25, 2019 at 4:32 PM Michael Miklavcic < > > > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > > > > > We're also not "incubator-metron" any longer. Do we require > > > any kind > > > > of > > > > > > > voting or +1 on that verification page to make corrections > to > > > it? > > > > > > > > > > > > > > On Thu, Apr 25, 2019 at 2:29 PM Michael Miklavcic < > > > > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > > > > > > > fyi, the steps in this doc have changed slightly per this > > > naming > > > > > > > > convention change as well - > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 25, 2019 at 1:25 PM Justin Leet < > > > justinjl...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > >> For everyone taking the time to validate and vote on the > > > RC, there > > > > > is > > > > > > a > > > > > > > >> caveat. The naming conventions for the two repos are now > > > aligned > > > > > > > >> (<repo_name>_<version>, instead of being '-' in the main > > > repo and > > > > > '_' > > > > > > in > > > > > > > >> the plugin repo) along with the location of the KEYS > file, > > > I have > > > > a > > > > > PR > > > > > > > out > > > > > > > >> to update the metron-rc-check script ( > > > > > > > >> https://github.com/apache/metron/pull/1394). > > > > > > > >> > > > > > > > >> This accounts for both of these changes, and should > allow > > > the > > > > script > > > > > > to > > > > > > > be > > > > > > > >> run normally. > > > > > > > >> > > > > > > > >> On Thu, Apr 25, 2019 at 3:22 PM Justin Leet < > > > > justinjl...@gmail.com> > > > > > > > >> wrote: > > > > > > > >> > > > > > > > >> > This is a call to vote on releasing Apache Metron > 0.7.1 > > > > > > > >> > > > > > > > > >> > Full list of changes in this release: > > > > > > > >> > > > > https://dist.apache.org/repos/dist/dev/metron/0.7.1-RC1/CHANGES > > > > > > > >> > The tag to be voted upon is: > > > > > > > >> > apache-metron_0.7.1-rc1 > > > > > > > >> > > > > > > > > >> > The source archives being voted upon can be found > here: > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/metron/0.7.1-RC1/apache-metron_0.7.1-rc1.tar.gz > > > > > > > >> > > > > > > > > >> > Other release files, signatures and digests can be > found > > > here: > > > > > > > >> > > > https://dist.apache.org/repos/dist/dev/metron/0.7.1-RC1/ > > > > > > > >> > > > > > > > > >> > The release artifacts are signed with the following > key: > > > > > > > >> > > https://dist.apache.org/repos/dist/release/metron/KEYS > > > > > > > >> > Please vote on releasing this package as Apache Metron > > > 0.7.1-RC1 > > > > > > > >> > > > > > > > > >> > When voting, please list the actions taken to verify > the > > > > release. > > > > > > > >> > > > > > > > > >> > Recommended build validation and verification > > > instructions are > > > > > > posted > > > > > > > >> > here: > > > > > > > >> > > > > > > > > > https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds > > > > > > > >> > > > > > > > > >> > This vote will be open for until 4pm EDT on Tuesday > > April > > > 30 > > > > 2019, > > > > > > to > > > > > > > >> > account for the weekend.. > > > > > > > >> > > > > > > > > >> > [ ] +1 Release this package as Apache Metron 0.7.1-RC1 > > > > > > > >> > > > > > > > > >> > [ ] 0 No opinion > > > > > > > >> > > > > > > > > >> > [ ] -1 Do not release this package because... > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
