[
https://issues.apache.org/jira/browse/METRON-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495203#comment-16495203
]
ASF GitHub Bot commented on METRON-1547:
----------------------------------------
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/1037
This can be tested both on via REST API and via the UI, although as noted
above there is UI instability for right now.
It should work for both ES and Solr (although right now Solr requires
`index` to be passed with requests, so make sure to add it in the following
steps. `index` will be the same as `sensorType`).
The examples tests are done with curl, but could also be done in Swagger.
## To setup Solr
```
sudo su -
export METRON_HOME=/usr/metron/0.4.3
cd ${METRON_HOME}/bin/
./install_solr.sh
./create_collection.sh bro
./create_collection.sh yaf
./create_collection.sh snort
./create_collection.sh error
./create_collection.sh metaalert
```
Edit the globe config at ${METRON_HOME}/config/zookeeper/global.json to
have "source.type.field"
, e.g.
"geo.hdfs.file" : "/apps/metron/geo/default/GeoLite2-City.mmdb.gz",
"source.type.field" : "source.type"
${METRON_HOME}/bin/zk_load_configs.sh -z node1:2181 -c GLOBAL -m PUSH -i
${METRON_HOME}/config/zookeeper/
Ensure the new config is found.
${METRON_HOME}/bin/zk_load_configs.sh -z node1:2181 -c GLOBAL -m DUMP
In Ambari:
Indexing -> Random Access Search Engine -> Solr
Restart Metron Indexing, Metron REST, and Metron Alerts UI
## Testing
Make sure to replace the guid and add index as needed throughout
### Get a GUID
```
curl -u user:password -X POST --header 'Content-Type: application/json'
--header 'Accept: application/json' -d '{
"fields": [
"guid"
],
"from": 0,
"indices": [
"bro"
],
"query": "*:*",
"size": 1
}' 'http://node1:8082/api/v1/search/search'
```
#### Sample Response
```
{
"total": 2120,
"results": [
{
"id": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"source": {
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412"
},
"score": 1,
"index": "bro_index_2018.05.15.16"
}
],
"facetCounts": null
}
```
### Create a new comment
```
curl -u user:password -X POST --header 'Content-Type: application/json'
--header 'Accept: */*' -d '{
"comment": "My Comment",
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro",
"timestamp": 1526401584951,
"username": "test_username"
}' 'http://node1:8082/api/v1/update/add/comment'
```
### Call findOne
```
curl -u user:password -X POST --header 'Content-Type: application/json'
--header 'Accept: application/json' -d '{
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro"
}' 'http://node1:8082/api/v1/search/findOne'
```
#### Response should contain a new comments field
```
"comments": [
{
"comment": "My Comment",
"username": "test_username",
"timestamp": 1526401584951
}
]
```
### Add another comment
```
curl -u user:password -X POST --header 'Content-Type: application/json'
--header 'Accept: */*' -d '{
"comment": "My Comment 2",
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro",
"timestamp": 1526401584955,
"username": "test_username_2"
}' 'http://node1:8082/api/v1/update/add/comment'
```
### Patch the comment with a new field
This ensures the raw form is properly translated during patch operations
(otherwise it can be mangled and not be readable later)
```
curl -u user:password -X PATCH --header 'Content-Type: application/json'
--header 'Accept: */*' -d '{
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"patch": [
{
"op": "add",
"path": "/project",
"value": "metron"
}
],
"sensorType": "bro"
}' 'http://node1:8082/api/v1/update/patch'
```
### Find it again
```
curl -u user:password -X POST --header 'Content-Type: application/json'
--header 'Accept: application/json' -d '{
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro"
}' 'http://node1:8082/api/v1/search/findOne'
```
Response should have both comments and the new field
```
"comments": [
{
"comment": "My Comment",
"username": "test_username",
"timestamp": 1526401584951
},
{
"comment": "My Comment 2",
"username": "test_username_2",
"timestamp": 1526401584955
}
]
```
### Remove comment
```
curl -u user:password -X POST --header 'Content-Type: application/json'
--header 'Accept: */*' -d '{
"comment": "My Comment",
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro",
"timestamp": 1526401584951,
"username": "test_username"
}' 'http://node1:8082/api/v1/update/remove/comment'
```
### Find it again.
```
curl -u user:password -X POST --header 'Content-Type: application/json'
--header 'Accept: application/json' -d '{
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro"
}' 'http://node1:8082/api/v1/search/findOne'
```
The comment should be removed, leaving something like
```
"comments": [
{
"comment": "My Comment 2",
"username": "test_username_2",
"timestamp": 1526401584955
}
]
```
If you repeat the remove, nothing should happen to the alert.
### Remove the remaining comment
```
curl -u user:password -X POST --header 'Content-Type: application/json'
--header 'Accept: */*' -d '{
"comment": "My Comment 2",
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"index": "bro_index_2018.05.15.16",
"sensorType": "bro",
"timestamp": 1526401584955,
"username": "test_username_2"
}' 'http://node1:8082/api/v1/update/remove/comment'
```
### Find the alert again.
It should no longer have comments.
If you repeat the remove, nothing should happen to the alert.
In the UI, similar steps can be performed to add comments to a particular
alert, as well as deleting them. The index can be checked to ensure the alert
itself has the correct results as above.
> Solr Comment Fields
> -------------------
>
> Key: METRON-1547
> URL: https://issues.apache.org/jira/browse/METRON-1547
> Project: Metron
> Issue Type: Sub-task
> Reporter: Justin Leet
> Assignee: Justin Leet
> Priority: Major
>
> Right now the Solr schemas don't have comment fields defined. It'll get
> caught by the catch all with ignored type not multivalued.
> ES just handles this correctly out of the box, but we'll need to take care of
> it in Solr and document the schema restriction.
> This actually is probably fairly problematic in comparison to ES. Solr
> doesn't support an easy way of doing a complex structure without doing
> something a bit weird (like parsing a string representation) or miserable
> (nested document).
> This will be incompatible with the current comment update system (just using
> the patch() functionality). Preferably we need to add a new REST endpoint for
> comments specifically so that we can handle it without the frontend knowing
> the backend system. This also involves adjusting the UI to use the new REST
> API, along with testing for both ES and Solr.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
