There are certainly some very interesting things we could do with a combination of the work we have in Metron and NiFi, especially around edge sensors.
Can I ask, Corey, Madhukar and any other on this, are there other related sources you’re wanting to bring in. Are you collecting NetFlow from multiple locations, shipping it across different networks for example? That would be a very good piece to do with NiFi. Do you need to collect all your NetFlow, or would you be interested in filtering it out at the NiFi stage? Would love to know and see what we can do to make NetFlow awesome in NiFi. Simon > On 17 Apr 2017, at 08:34, Joe Witt <joe.w...@gmail.com> wrote: > > I've seen several over the past couple years used in NiFi. I'd bet > with a little bit of work and collab with Apache Metron we could have > one quickly should someone be in a position to contribute. > > On Mon, Apr 17, 2017 at 11:23 AM, Aldrin Piri <aldrinp...@gmail.com> wrote: >> Hi Corey, >> >> Does not look like there has been an implementation at this point in time. >> There is a JIRA that is tracking the request of this thread: >> https://issues.apache.org/jira/browse/NIFI-2904 >> >> On Mon, Apr 17, 2017 at 10:57 AM, Corey Flowers <cflow...@onyxpoint.com> >> wrote: >>> >>> Good morning everyone, >>> >>> Was there ever a netflow parser processor built? I am currently >>> working on a netflow issue and have seen several people discuss netflow >>> parsing in threads but I haven't seen any information about the release of a >>> netflow processor. The version I am currently working with is v9. >>> >>> Thanks! >>> >>> On Mon, Aug 8, 2016 at 10:00 AM, Madhukar Thota <madhukar.th...@gmail.com> >>> wrote: >>>> >>>> Hi Joe, >>>> >>>> we have bunch of cisco router that has netflow feature built in. Netflow >>>> allows us to collect the network traffic from the devices and able to send >>>> them to udp destination for processing. As the data comes in raw, we need >>>> to >>>> parse the incoming data and do transformation and send to them hdfs, kafka >>>> or elasticseach or some other destination for analytics. >>>> >>>> Today we are using logstash as netflow collector and able to do >>>> transformation and write them into elasticsearch for visualization. As we >>>> are moving all the logstash processing work to nifi, we want to move the >>>> netflow parsing to nifi too. >>>> >>>> https://github.com/logstash-plugins/logstash-codec-netflow >>>> >>>> http://blogs.cisco.com/security/step-by-step-setup-of-elk-for-netflow-analytics >>>> >>>> https://streamsets.com/documentation/datacollector/latest/help/#Origins/UDP.html#concept_rst_2y5_1s >>>> >>>> >>>> -Madhu >>>> >>>> On Mon, Aug 8, 2016 at 9:42 AM, Joe Witt <joe.w...@gmail.com> wrote: >>>>> >>>>> Hello >>>>> >>>>> There are no processors included in the apache release that >>>>> specifically operate on netflow so you'd need to have a custom >>>>> processor to deal with it until one is included. >>>>> >>>>> Netflow is often flowing through NiFi typically for things like >>>>> content merging and loading into HDFS. Parsing is a good use case and >>>>> presumably after that you'd want to make some routing decisions or do >>>>> some sort of enrichment? Can you describe in more detail what you'd >>>>> like to be able to accomplish in NiFi and what systems it would >>>>> deliver the netflow to? Also, what type of Netflow is of interest (it >>>>> can be frustratingly proprietary)? >>>>> >>>>> Thanks >>>>> Joe >>>>> >>>>> On Mon, Aug 8, 2016 at 9:27 AM, Madhukar Thota >>>>> <madhukar.th...@gmail.com> wrote: >>>>>> Is there any Processor available for Netflow? If not what is the best >>>>>> way to >>>>>> get Netflow data parsed using nifi? >>>>>> >>>>>> >>>> >>>> >>> >>> >>> >>> -- >>> Corey Flowers >>> Vice President, Onyx Point, Inc >>> (410) 541-6699 >>> cflow...@onyxpoint.com >>> >>> -- This account not approved for unencrypted proprietary information -- >> >>
