Hi,
Thanks a lot for the inputs.
@Stefan: We don’t have ranger setup for access permissions.
@Ryan: Below are the issues I observed from your suggestion
For the search request in network tab, I see the response is
“500 Internal Server Error”
In the request payload I see all the params are emtpy as below.
{indices: [], facetFields: [], query: "*",
from: 0, size: 25}
When I try to run the curl command, I am getting unauthorized error as
below. How can I pass credentials for authentication along with the request or
how can I authenticate via curl command and store the jessionid as cookie and
pass through your curl command?
{"timestamp":"2018-08-23
11:13:22","status":401,"error":"Unauthorized","message":"Unauthorized","path":"/api/v1/sensor/indexing/config/list/indices/solr"}
Thanks,
Anil.
From: Ryan Merriman [mailto:[email protected]]
Sent: Thursday, August 23, 2018 3:41 AM
To: [email protected]
Subject: Re: Unable to see alerts in metron alert UI with solr
The first thing I would do is figure out which indices are included in the
Alerts UI. If you open the network tab in your browser there should be a
"search" request sent to REST. There is an "indices" property that will have a
list of indices include in the search. If it's empty ("[]") then this call
should tell you what the default indices are:
curl -X GET --header 'Accept: application/json' 'http://{rest
host}:8082/api/v1/sensor/indexing/config/list/indices/solr'
Verify your collection is in the list and we can go from there.
On Wed, Aug 22, 2018 at 4:47 PM Stefan Kupstaitis-Dunkler
<[email protected]<mailto:[email protected]>> wrote:
Hey Anil,
do you use any kind of policies to prevent access by the metron user? Do you
use Ranger to manage access policies?
The user running the Metron REST service (normally "metron") needs to have
access to all collections.
Check the (Ranger) audit log to see if access to the collections is
denied/allowed.
If you use Ranger + Solr with Metron you might run into a current Solr plugin
bug. I've run into that a few weeks ago and provided a workaround here:
https://community.hortonworks.com/articles/203326/searching-in-multiple-collections-in-one-query-wit.html<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fcommunity.hortonworks.com%2farticles%2f203326%2fsearching-in-multiple-collections-in-one-query-wit.html&c=E,1,9dD6y1u3ennoQvuZCZ45l9VNmxiTrI6YNYlhGiC0_1XXX0pMj7qHTj6yBi6drFMrosDMlFfiJthaQqZTUjr6MVXrJwuRPuh_eNO6-s9QM6lnTnhpPaw,&typo=1>
Best,
Stefan
On Wed, Aug 22, 2018, 18:40 Anil Donthireddy
<[email protected]<mailto:[email protected]>> wrote:
Yes, necessary two fields are set properly as below
"source.type.field" : "source.type",
"threat.triage.score.field" : "threat.triage.score"
From: Anand Subramanian
[mailto:[email protected]<mailto:[email protected]>]
Sent: Wednesday, August 22, 2018 10:07 PM
To: [email protected]<mailto:[email protected]>
Subject: Re: Unable to see alerts in metron alert UI with solr
Hey Anil,
You might also want to check if the “source.type.field” is set to “source.type”
(and NOT source:type) in global config.
See:
https://github.com/apache/metron/blob/master/metron-interface/metron-alerts/README.md#global-configuration-properties
HTH,
Anand
From: Anil Donthireddy
<[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Wednesday, August 22, 2018 at 10:04 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: RE: Unable to see alerts in metron alert UI with solr
Hi Stefan,
I can see the events being written to solr collection. There are no issues upto
storing data to Solr. There are events which have is_alert=true. But in the
metron alerts UI I dint see any events.
Thanks,
Anil.
From: Stefan Kupstaitis-Dunkler [mailto:[email protected]]
Sent: Wednesday, August 22, 2018 9:52 PM
To: [email protected]<mailto:[email protected]>
Subject: Re: Unable to see alerts in metron alert UI with solr
Hi Anil,
The alerts UI just queries your Solr collections and displays them
Things you could do:
• You could check the collection the event is supposed to be directly
• If it’s not there you could check the Solr “error” collection.
• Use the Kafka console consumer and
o check the parser topic (you define the name in the management ui) if events
are being written to it
o check the “enrichments” topic if events are being written to it
o check the “indexing” topic if events are being written to it.
Hope that helps as a start.
Best,
Stefan
From: Anil Donthireddy
<[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Wednesday, 22. August 2018 at 18:07
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Unable to see alerts in metron alert UI with solr
Hi,
I have created a collection in solr for my source as documented in metron-slor
readme in git to enable solr and writing data to solr. I see the data is being
loaded to Solr for my new source and is_alert is set to tru. But still I am not
seeing any data in metron alerts UI.
I checked the metron alerts UI logs, but could not find any error logs. I am
blocked to proceed further to display alerts in the metron alerts ui. May I get
any suggestions to debug the issue.
Thanks,
Anil.
--
Stefan Kupstaitis-Dunkler
https://datahovel.com/<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fdatahovel.com%2f&c=E,1,rgB6RnyVWsWRLyz4Pfv-ZJ0kCNc2qsDK9f0qPgKhSLPGI8cUtFb3JzjKtCJ0C5P5533BhkBS_k9oAum3K-N7e8fYNQHryxJrRnCjR5uHcw,,&typo=1>
https://www.meetup.com/Hadoop-User-Group-Vienna/
https://twitter.com/StefanDunkler
