I sent a random message to that kafka topic and got this [image: Inline image 1]
I guess this is because I am not following the format of message I should send? Like those snort logs you showed. On Mon, Oct 30, 2017 at 5:24 PM, zeo...@gmail.com <zeo...@gmail.com> wrote: > They need to meet the format of the logs I sent earlier. Look into the > snort output options - may require you rerun snort, depending on your > situation > > Jon > > On Mon, Oct 30, 2017, 06:53 Syed Hammad Tahir <mscs16...@itu.edu.pk> > wrote: > >> Yes, I have converted them to text but those logs are simply captured >> packet headers over the local network. Now I just push them via that kafka >> producer command under topic name of snort and they will be visible in >> metron? >> >> On Mon, Oct 30, 2017 at 2:41 PM, zeo...@gmail.com <zeo...@gmail.com> >> wrote: >> >>> You need text logs. Here's an example of some properly formatted logs - >>> https://raw.githubusercontent.com/apache/metron/master/metron- >>> deployment/roles/sensor-stubs/files/snort.out >>> >>> Jon >>> >>> On Mon, Oct 30, 2017, 01:34 Syed Hammad Tahir <mscs16...@itu.edu.pk> >>> wrote: >>> >>>> I have found the kafka-console-producer.sh but I need to know how to >>>> make it read snort.log (tcp dump format) file. May be I am missing >>>> something in the plain sight but it would be awsome if you tell me that. >>>> >>>> Regards. >>>> >>>> On Fri, Oct 27, 2017 at 5:09 PM, zeo...@gmail.com <zeo...@gmail.com> >>>> wrote: >>>> >>>>> On the 25th I said: >>>>> >>>>> It should be in /usr/hdp/current/kafka-broker/bin/ or similar >>>>> (from memory) on node1, assuming you are running full dev. >>>>> >>>>> Jon >>>>> >>>>> >>>>> Jon >>>>> >>>>> On Fri, Oct 27, 2017 at 6:25 AM Syed Hammad Tahir < >>>>> mscs16...@itu.edu.pk> wrote: >>>>> >>>>>> snort logs are in tcp dump format. I may have to convert them. >>>>>> >>>>>> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic >>>>>> test >>>>>> >>>>>> How to give file name or path in this command? >>>>>> >>>>>> On Fri, Oct 27, 2017 at 2:53 PM, zeo...@gmail.com <zeo...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> If you have text snort logs you can use Apache nifi or the Kafka >>>>>>> producer script as described in step 4 here[1] to push them to Metron's >>>>>>> snort topic. You may also want to look at this [2]. >>>>>>> >>>>>>> 1: https://kafka.apache.org/quickstart >>>>>>> 2: https://stackoverflow.com/questions/38701179/kafka- >>>>>>> console-producer-and-bash-script >>>>>>> >>>>>>> Jon >>>>>>> >>>>>>> On Fri, Oct 27, 2017, 02:15 Syed Hammad Tahir <mscs16...@itu.edu.pk> >>>>>>> wrote: >>>>>>> >>>>>>>> Hello everyone, >>>>>>>> >>>>>>>> I have run snort independently on vagrant ssh and dumped the logs >>>>>>>> in tcpdump format. Now I want to bring them to metron to play with >>>>>>>> them a >>>>>>>> bit. Some of you already replied me with some solutions but thats lost >>>>>>>> in >>>>>>>> the inbox somewhere and engulfed by the elasticsearhc issue that I had. >>>>>>>> Please give me an easy to understand this solution for this problem. >>>>>>>> >>>>>>>> Regards. >>>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Jon >>>>>>> >>>>>> >>>>>> -- >>>>> >>>>> Jon >>>>> >>>> >>>> -- >>> >>> Jon >>> >> >> -- > > Jon >
