Are we missing any configuration? Initially elastic search was down. We figured out the issue and fixed it .Now elastic search is up . We restarted metron indexing but still those indices not created. So we created it manually.Do we have to change any parser configuration . How logs will flow into metron alerts dashboard and kibana dashboard..what is the required congratulation
On Fri, Apr 5, 2019, 11:52 PM Hema malini <[email protected]> wrote: > Sample messages flown in indexing topic > {"msg":"'snort test alert'","parallelenricher.splitter.end.ts":" > 1554384505264","sig_rev":"0","ip_dst_port":"50183","ethsrc": > "08:00:27:E8:B0:7A","threat.triage.rules.0.comment":null," > tcpseq":"0x8DF34F4B","threat.triage.score":10.0,"dgmlen":"52","adapter. > hostfromjsonlistadapter.end.ts":"1554384503452","adapter. > geoadapter.begin.ts":"1554384503452","tcpwindow":" > 0x1F5","parallelenricher.splitter.begin.ts":" > 1554384505264","threat.triage.rules.0.score":"10","tcpack":" > 0x836687BD","protocol":"TCP","ip_dst_addr":"192.168.66.1"," > original_string":"01\/11\/17-20:53:16.104984 ,1,999158,0,\"'snort test > alert'\",TCP,192.168.66.121,8080,192.168.66.1,50183,08:00: > 27:E8:B0:7A,0A:00:27:00:00:00,0x42,***A****,0x8DF34F4B, > 0x836687BD,,0x1F5,64,0,62040,52,53248,,,,","parallelenricher.enrich.end. > ts":"1554384505342","threat.triage.rules.0.reason":null," > tos":"0","adapter.hostfromjsonlistadapter.begin.ts":"1554384503452","id":" > 62040","ip_src_addr":"192.168.66.121","timestamp": > 1484148196104,"ethdst":"0A:00:27:00:00:00","threat.triage.rules.0.name > ":null,"is_alert":"true","parallelenricher.enrich.begin.ts":" > 1554384505264","ttl":"64","source.type":"snort","adapter. > geoadapter.end.ts":"1554384503453","ethlen":"0x42" > ,"iplen":"53248","adapter.threatinteladapter.begin.ts":" > 1554384505264","ip_src_port":"8080","tcpflags":"***A****"," > guid":"2f6f3f3c-7739-47fe-aa04-3c62425fbcbf","sig_id":" > 999158","sig_generator":"1"} > > > On Fri, Apr 5, 2019, 11:43 PM Hema malini <[email protected]> wrote: > >> Yes I am getting messages >> >> On Fri, Apr 5, 2019, 11:17 PM Michael Miklavcic < >> [email protected]> wrote: >> >>> Do you get 10 records output to the CLI when you run the following? >>> >>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper >>> $ZOOKEEPER --topic indexing --from-beginning --max-messages 10 >>> >>> >>> On Fri, Apr 5, 2019 at 11:38 AM Hema malini <[email protected]> >>> wrote: >>> >>>> We verified it in Storm ui and in Storm topology logs >>>> >>>> On Fri, Apr 5, 2019, 10:53 PM Michael Miklavcic < >>>> [email protected]> wrote: >>>> >>>>> How did you validate the logs are making it to the indexing topology? >>>>> >>>>> On Fri, Apr 5, 2019 at 8:12 AM Hema malini <[email protected]> >>>>> wrote: >>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> >>>>>> >>>>>> We have installed Metron 0.7.1 in centos 7 using Amabari.Using Nifi >>>>>> we sent the sample snort logs copied from metron git repo to snort kafka >>>>>> topic.We did the same for bro topic.Logs are getting parsed and reached >>>>>> indexing topology . Elastic search indices are not getting created though >>>>>> we gave elastic search template install from ambari. So manually created >>>>>> the elastic search index using template available in metron repo. >>>>>> Though elastic search index is present , data from indexing toplogy >>>>>> neither >>>>>> reached elastic search nor hdfs path .There are no errors in storm >>>>>> toplogy >>>>>> logs.We could see the sample log in Metron management ui. How we can send >>>>>> the logs to alerts ui and kibana dashboard. In kibana dashboard we could >>>>>> see two dashboards - Metron-Dashboard,Metron-Error-Dashboard created but >>>>>> with no data. Elasticsearch health is yellow and we are able to insert >>>>>> data >>>>>> via rest call. Any documentation on sending the smaple snort logs to >>>>>> metron >>>>>> alerts ui will be helpful . Any configuration from metron management ui >>>>>> is >>>>>> required to pass it to alerts –ui >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Thanks and Regards >>>>>> >>>>>> Hema >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>
