Thanks Luca! Just tried the latest nightly snapshot.
This is really looking promising, we have 16800 flows/s now instead of 5xx /s (for whatever reason). Looking forward to production release of the kafka integration. PS: Any plans to be able to use more than 1 kafka broker and also a port ? Von: <[email protected]<mailto:[email protected]>> im Auftrag von Luca Deri Antworten an: "[email protected]<mailto:[email protected]>" Datum: Mittwoch, 30. September 2015 15:00 An: "[email protected]<mailto:[email protected]>" Betreff: Re: [Ntop-misc] nprobe and kafka? Florian the kafka integration is already implemented in the development package. deri@ubuntu14 202> ./nprobe -h|grep kafka --kafka <brokerIP>:<topic>:<partition> | Send flows to the specified Apache Kafka broker. Example --kafka localhost:test:1 On 09/30/2015 02:21 PM, Florian Pieper wrote: Hi Luca, I am currently trying to use nprobe in collector mode and forward JSON formatted flows to flume (netcat) Which in return forwards the data to kafka. The export queue in nprobe is filling up (already increased -w) and my guess is that there is some kind of Bottleneck between nprobe and flume. if it increases it is because the collector is slow and -w won't help You are saying that there is some development going on for a direct kafka integration in nprobe? Can you let me know where I can download this special version and when we can expect it to be in the Production branch? It is production ready but it is in the dev packages. Please test it and report us bugs. Regards Luca Thanks, Florian From: <<mailto:[email protected]>[email protected]<mailto:[email protected]>> on behalf of Luca Deri <[email protected]<mailto:[email protected]>> Reply-To: "<mailto:[email protected]>[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Tuesday, September 1, 2015 at 10:52 PM To: "<mailto:[email protected]>[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: [Ntop-misc] nprobe and kafka? Manny we have added kafka support on one of our development prototypes so movign to the official nprobe should not be too difficult. The performance is similar to the ZMQ or elasticsearch implementation, so considered the JSON conversion is at least 1k flows/sec Luca On 01 Sep 2015, at 23:20, Manny Veloso <[email protected]<mailto:[email protected]>> wrote: Hi! I’m looking to use nprobe as a bridge into kafka. In the splunk app nprobe just sends data into splunk. Is that basically the same configuration as a kafka install? Also, what kind of throughput can I expect out of nprobe? -- Manny Veloso Sr. Solutions Engineer Smartrg.com<http://Smartrg.com> _______________________________________________ Ntop-misc mailing list [email protected]<mailto:[email protected]> http://listgateway.unipi.it/mailman/listinfo/ntop-misc _______________________________________________ Ntop-misc mailing list [email protected]<mailto:[email protected]>http://listgateway.unipi.it/mailman/listinfo/ntop-misc
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
