Okay, so is it starting as a new architecture?
On 2021/02/03 01:28:21, Simon Elliston Ball <[email protected]> wrote: > We’re also replacing much of the functionality of Metron with a series of > Apache Flink based streaming components in a number of installations. It > makes for a composable approach, and borrows from elements of the Metron > architecture, while using more efficient formats like Apache Avro instead > of JSON to reduce Kafka consumption and increase performance. Using Flink > also allows for more efficient aggregation and sql based rules. > > It’s much more of a custom solution than the generic project Metron took > on, but seems to be working well for many of the same log related use cases. > > Simon > > On Wed, 3 Feb 2021 at 00:08, Alex Scammon <[email protected]> wrote: > > > Hey there Jack, > > > > We were also disappointed that Metron was shuttered. But we've seen it as > > an opportunity to continue an internal project which builds on top of > > Metron. Our goal is to make our project open source as a potential > > successor to Metron. > > > > We're maybe a month or two away from making it public, but we'd love some > > eyes on it before we take that step if you're interested in taking a look. > > > > Since it builds on Metron, a lot of the core architecture remains > > familiar. Java, Kafka, Storm, etc.... Hopefully, that presents a familiar > > ecosystem for folks who are currently using Metron. For improvements, we > > focused on: > > > > - Ensuring that simple configuration mistakes don't bring down the > > whole pipeline > > - A git-based approval workflow for rules updates (approvals and an > > audit trail are important for us) > > - An improved, modern-looking UI in Angular > > - Easier installation steps > > > > Let me know if you're interested in discussing more -- I'd be interested > > to hear whether there are any particulars about the models you're running > > that we should take into consideration. > > > > Cheers, > > > > Alex Scammon > > Head of Open Source Development > > G-Research > > gresearch.co.uk > > > > > > On Tue, Feb 2, 2021 at 1:18 AM Jack Roberts <[email protected]> wrote: > > > >> Hi Metron community, > >> > >> > >> > >> I recently started to explore Metron as part of a research project I’m > >> involved in, but I’ve just seen the unfortunate news that the project is > >> being “moved to the Attic”. I’d be very grateful if anyone could help to > >> clarify the following: > >> > >> > >> > >> - Is there likely to be any continued development of Metron outside > >> of Apache? > >> - Are there any alternatives to Metron that people in this community > >> would recommend? In particular, we’re looking for something open source > >> that we can deploy ourselves, and with the functionality to > >> straightforwardly integrate our own machine learning models for anomaly > >> detection/similar. > >> > >> > >> > >> Many thanks and best wishes, > >> > >> Jack > >> The Alan Turing Institute is a limited liability company, registered in > >> England with registered number 09512457 with registered offices at British > >> Library, 96 Euston Road, London, England, NW1 2DB > >> <https://www.google.com/maps/search/96+Euston+Road,+London,+England,+NW1+2DB?entry=gmail&source=g>. > >> We are also a charity registered in England with charity number 1162533. > >> DISCLAIMER: Although we have taken reasonable precautions to ensure the > >> completeness and accuracy of this e-mail, transmission cannot be guaranteed > >> to be secure or error-free as information could be intercepted, corrupted, > >> lost, destroyed, arrive late or be incomplete. If you receive a suspicious > >> or unexpected email from us, or purporting to have been sent on our behalf, > >> particularly containing different bank details, please do not reply to the > >> email, click on any links, open any attachments, or comply with any > >> instructions contained within it. Our Transparency Notice found here - > >> https://www.turing.ac.uk/transparency-notice sets out how and why we > >> collect, store, use and share your personal data and it explains your > >> rights and how to raise concerns with us. > >> > > -- > -- > simon elliston ball > @sireb >
