After recreating the index, now we are able to visualize the data in kibana metron dashboard. How we can pass alerts to metron alerts UI. Currently there is no data in alerts UI. How.to configure the logs as alerts
On Sat, Apr 6, 2019, 9:21 PM Hema malini <[email protected]> wrote: > Sorry for the typo. Can you please help with the required configuration. > > On Sat, Apr 6, 2019, 5:39 PM Hema malini <[email protected]> wrote: > >> Are we missing any configuration? Initially elastic search was down. We >> figured out the issue and fixed it .Now elastic search is up . We restarted >> metron indexing but still those indices not created. So we created it >> manually.Do we have to change any parser configuration . How logs will flow >> into metron alerts dashboard and kibana dashboard..what is the required >> congratulation >> >> On Fri, Apr 5, 2019, 11:52 PM Hema malini <[email protected]> >> wrote: >> >>> Sample messages flown in indexing topic >>> {"msg":"'snort test alert'","parallelenricher.splitter.end.ts":" >>> 1554384505264","sig_rev":"0","ip_dst_port":"50183","ethsrc": >>> "08:00:27:E8:B0:7A","threat.triage.rules.0.comment":null," >>> tcpseq":"0x8DF34F4B","threat.triage.score":10.0,"dgmlen":"52","adapter. >>> hostfromjsonlistadapter.end.ts":"1554384503452","adapter. >>> geoadapter.begin.ts":"1554384503452","tcpwindow":" >>> 0x1F5","parallelenricher.splitter.begin.ts":" >>> 1554384505264","threat.triage.rules.0.score":"10","tcpack":" >>> 0x836687BD","protocol":"TCP","ip_dst_addr":"192.168.66.1"," >>> original_string":"01\/11\/17-20:53:16.104984 ,1,999158,0,\"'snort test >>> alert'\",TCP,192.168.66.121,8080,192.168.66.1,50183,08:00: >>> 27:E8:B0:7A,0A:00:27:00:00:00,0x42,***A****,0x8DF34F4B, >>> 0x836687BD,,0x1F5,64,0,62040,52,53248,,,,","parallelenricher.enrich.end. >>> ts":"1554384505342","threat.triage.rules.0.reason":null," >>> tos":"0","adapter.hostfromjsonlistadapter.begin. >>> ts":"1554384503452","id":"62040","ip_src_addr":"192.168. >>> 66.121","timestamp":1484148196104,"ethdst":"0A:00:27:00:00:00"," >>> threat.triage.rules.0.name":null,"is_alert":"true","parallelenricher. >>> enrich.begin.ts":"1554384505264","ttl":"64"," >>> source.type":"snort","adapter.geoadapter.end.ts":" >>> 1554384503453","ethlen":"0x42","iplen":"53248","adapter. >>> threatinteladapter.begin.ts":"1554384505264","ip_src_port":" >>> 8080","tcpflags":"***A****","guid":"2f6f3f3c-7739-47fe- >>> aa04-3c62425fbcbf","sig_id":"999158","sig_generator":"1"} >>> >>> >>> On Fri, Apr 5, 2019, 11:43 PM Hema malini <[email protected]> >>> wrote: >>> >>>> Yes I am getting messages >>>> >>>> On Fri, Apr 5, 2019, 11:17 PM Michael Miklavcic < >>>> [email protected]> wrote: >>>> >>>>> Do you get 10 records output to the CLI when you run the following? >>>>> >>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh >>>>> --zookeeper $ZOOKEEPER --topic indexing --from-beginning --max-messages 10 >>>>> >>>>> >>>>> On Fri, Apr 5, 2019 at 11:38 AM Hema malini <[email protected]> >>>>> wrote: >>>>> >>>>>> We verified it in Storm ui and in Storm topology logs >>>>>> >>>>>> On Fri, Apr 5, 2019, 10:53 PM Michael Miklavcic < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> How did you validate the logs are making it to the indexing topology? >>>>>>> >>>>>>> On Fri, Apr 5, 2019 at 8:12 AM Hema malini <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> We have installed Metron 0.7.1 in centos 7 using Amabari.Using Nifi >>>>>>>> we sent the sample snort logs copied from metron git repo to snort >>>>>>>> kafka >>>>>>>> topic.We did the same for bro topic.Logs are getting parsed and reached >>>>>>>> indexing topology . Elastic search indices are not getting created >>>>>>>> though >>>>>>>> we gave elastic search template install from ambari. So manually >>>>>>>> created >>>>>>>> the elastic search index using template available in metron repo. >>>>>>>> Though elastic search index is present , data from indexing toplogy >>>>>>>> neither >>>>>>>> reached elastic search nor hdfs path .There are no errors in storm >>>>>>>> toplogy >>>>>>>> logs.We could see the sample log in Metron management ui. How we can >>>>>>>> send >>>>>>>> the logs to alerts ui and kibana dashboard. In kibana dashboard we >>>>>>>> could >>>>>>>> see two dashboards - Metron-Dashboard,Metron-Error-Dashboard created >>>>>>>> but >>>>>>>> with no data. Elasticsearch health is yellow and we are able to insert >>>>>>>> data >>>>>>>> via rest call. Any documentation on sending the smaple snort logs to >>>>>>>> metron >>>>>>>> alerts ui will be helpful . Any configuration from metron management >>>>>>>> ui is >>>>>>>> required to pass it to alerts –ui >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks and Regards >>>>>>>> >>>>>>>> Hema >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>
