The “top three” changed, so here are my preferred references for writing new parsers:
* https://cwiki.apache.org/confluence/display/METRON/2016/04/25/Metron+Tutorial+-+Fundamentals+Part+1%3A+Creating+a+New+Telemetry * https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source * https://metron.apache.org/current-book/metron-platform/metron-parsers/index.html From: Matt Foley <mfo...@hortonworks.com> on behalf of Matt Foley <ma...@apache.org> Date: Monday, July 31, 2017 at 2:39 PM To: "user@metron.apache.org" <user@metron.apache.org> Subject: Re: Integration of Honeeepi(honeypot sensor) with Metron Hi Naveen, Does Honeeepi produce a stream of logs and/or alerts, that you would like to process? If not, you’ll need to define a “sensor” of sorts that will tell you when something interesting happens (or is happening) with the honeypot. Metron does not help with that, although it can help compare normative with aberrational event streams, thereby identifying what is “interesting”, if Honeeepi itself does not do that. The integration point with Metron will be the message stream from Honeeepi or that Honeeepi sensor, preferably piped into Kafka. Next you need a parser for the logs from Kafka. You may be able to write a Grok script for our generic Grok parser, otherwise you can write a Metron Parser module in Java. Parsers are in the process of becoming plug-ins for Metron, but for now, the current way of creating new parsers can be found in the top three results when you google “apache metron writing a new parser”. Parsers convert messages of whatever format into a standard JSON format, which the rest of Metron knows how to deal with. Now you’ve got your “integration”. You still need to decide what to do with the message stream. If you need to identify “interesting” vs “not interesting” events, you might plug in an ML model as one of your enrichers. When you can filter for interesting events, you can “enrich” them by raising select info in the message body into the meta-data, or adding new meta-data based on associational lookups of existing fields. Then you add Threat Intel and do threat recognition, and feed that into your Indexing and Alerting sub-systems. That’s Metron in a thimble :-) Suggest you read the entire site-book of Metron development information at https://metron.apache.org/current-book/index.html , especially the articles with architecture diagrams at * https://metron.apache.org/current-book/metron-platform/metron-parsers/index.html * https://metron.apache.org/current-book/metron-platform/metron-enrichment/index.html * https://metron.apache.org/current-book/metron-platform/metron-indexing/index.html * https://metron.apache.org/current-book/metron-analytics/metron-maas-service/index.html and the stuff about Profiling and Statistics at * https://metron.apache.org/current-book/metron-analytics/metron-profiler/index.html * https://metron.apache.org/current-book/metron-analytics/metron-profiler-client/index.html * https://metron.apache.org/current-book/metron-analytics/metron-statistics/index.html Hope this helps, --Matt From: Naveen Narayanasamy <naveennarayanas...@cmail.carleton.ca> Reply-To: "user@metron.apache.org" <user@metron.apache.org> Date: Monday, July 31, 2017 at 8:21 AM To: "user@metron.apache.org" <user@metron.apache.org> Subject: Integration of Honeeepi(honeypot sensor) with Metron Hello all, I would like to know the possibility of integrating Honeeepi omponents(Cowrie and Dionaea) with Apache Metron.As there are limited information available online,I would also like to know the different integration procedures that can be tried. For more info about the honeeepi, please visit the link below https://redmine.honeynet.org/projects/honeeepi/wiki Wiki - Honeeepi - Honeynet Project Redmine redmine.honeynet.org Introduction¶ This project is about setting up honeypots with Raspberry Pi - a credit card sized ARM Linux box. !! Raspberry Pi¶ The Raspberry Pi is a credit sized ... Any response will be appreciated!! Naveen
