Aris, I am not aware of an out of the box tool for Pcap->Kafka ingestion (in my case back then we wrote our own). Maybe others know.
On Monday, October 10, 2016, Aris Risdianto <[email protected]> wrote: > Thank you for answer Michael. > > Actually, I have made a simple producer from Pcap to Kafka. Since it is not > structured, so it is difficult for further processing by a consumer. But, I > will take a look at Avro as you mentioned. > > I just wondering, if there are any proper implementation for this > requirement, because I couldn't find any tool in the kafka ecosystem page. > > https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem > > > Best Regards, > Aris. > > > On Mon, Oct 10, 2016 at 6:55 PM, Michael Noll <[email protected] > <javascript:;>> wrote: > > > Aris, > > > > even today you can already use Kafka to deliver Netflow/Pcap/etc. > messages, > > and people are already using it for that (I did that in previous projects > > of mine, too). > > > > Simply encode your Pcap/... messages appropriately (I'd recommend to > take a > > look at Avro, which allows you to structure your data similar to e.g. > > Pcap's native format [1]), and then write the encoded messages to Kafka. > > Your downstream applications can then read the encoded messages back from > > Kafka, decode, and commence processing. > > > > That was a brief summary to get you started, feel free to take a look at > > the Apache Kafka docs at kafka.apache.org and/or ask further questions > > here. > > > > -Michael > > > > > > > > > > [1] https://wiki.wireshark.org/Development/LibpcapFileFormat > > > > On Mon, Oct 10, 2016 at 11:19 AM, Aris Risdianto <[email protected] > <javascript:;>> wrote: > > > > > Hello, > > > > > > > > > Is there any plan or implementation to use Kafka for delivering > > > sFlow/NetFlow/Pcap messages? > > > > > > > > > Best Regards, > > > Aris. > > > > > > -- *Michael G. Noll* Product Manager | Confluent +1 650 453 5860 | @miguno <https://twitter.com/miguno> Follow us: Twitter <https://twitter.com/ConfluentInc> | Blog <http://www.confluent.io/blog>
