[
https://issues.apache.org/jira/browse/METRON-508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056102#comment-16056102
]
ASF GitHub Bot commented on METRON-508:
---------------------------------------
Github user JonZeolla commented on a diff in the pull request:
https://github.com/apache/metron/pull/586#discussion_r123037274
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template
---
@@ -238,6 +238,538 @@
"qtype_name": {
"type": "string",
"index": "not_analyzed"
+ },
+ "analyzer": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "failure_reason": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "user": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "password": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "command": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "arg": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "mime_type": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "file_size": {
+ "type": "long"
+ },
+ "reply_code": {
+ "type": "integer"
+ },
+ "reply_msg": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "data_channel:passive": {
+ "type": "boolean"
+ },
+ "data_channel:orig_h": {
+ "type": "ip"
+ },
+ "data_channel:resp_h": {
+ "type": "ip"
+ },
+ "data_channel:resp_p": {
+ "type": "integer"
+ },
+ "cwd": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "passive": {
+ "type": "boolean"
+ },
+ "capture_password": {
+ "type": "boolean"
+ },
+ "fuid": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "conn_uids": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "source": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "depth": {
+ "type": "integer"
+ },
+ "analyzers": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "filename": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "duration": {
+ "type": "float"
+ },
+ "local_orig": {
+ "type": "boolean"
+ },
+ "is_orig": {
+ "type": "boolean"
+ },
+ "seen_bytes": {
+ "type": "long"
+ },
+ "total_bytes": {
+ "type": "long"
+ },
+ "missing_bytes": {
+ "type": "long"
+ },
+ "overflow_bytes": {
+ "type": "long"
+ },
+ "timedout": {
+ "type": "boolean"
+ },
+ "parent_fuid": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "md5": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "sha1": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "sha256": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "port_num": {
+ "type": "integer"
+ },
+ "subject": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "issuer_subject": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "serial": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "helo": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "mailfrom": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "rcptto": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "date": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "from": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "to": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "reply_to": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "msg_id": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "in_reply_to": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "x_originating_ip": {
+ "type": "ip"
+ },
+ "first_received": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "second_received": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "last_reply": {
+ "type": "string",
+ "analyzer": "simple"
+ },
+ "path": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "tls": {
+ "type": "boolean"
+ },
+ "fuids": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "is_webmail": {
+ "type": "boolean"
+ },
+ "version": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "cipher": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "curve": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "server_name": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "resumed": {
+ "type": "boolean"
+ },
+ "last_alert": {
+ "type": "string",
+ "index": "not_analyzed"
+ },
+ "next_protocol": {
--- End diff --
Because you can create arbitrary fields in log sources, I decided to only
support the default-on fields with this first PR.
`proto` exists
[here](https://github.com/JonZeolla/metron/blob/7e761480c9749d67acfa7de538f54eee96dcba05/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template#L215-L218)
(i.e. it already existed in another bro log, so in these scenarios I made sure
that the type works for both - a more preferred alternative would be to break
out individual bro logs into separate indexes), `disabled_aids` is an [internal
only
field](https://github.com/bro/bro/blob/57da2d091b30aad52d52fce8018feeb2cdf8ff1f/scripts/base/frameworks/dpd/main.bro#L25-L27),
and `packet_segment` was explicitly omitted, as it requires that
`policy/frameworks/dpd/packet-segment-logging.bro` be loaded.
> Expand Elasticsearch templates to support the standard bro logs
> ---------------------------------------------------------------
>
> Key: METRON-508
> URL: https://issues.apache.org/jira/browse/METRON-508
> Project: Metron
> Issue Type: Sub-task
> Reporter: Jon Zeolla
> Assignee: Jon Zeolla
> Priority: Minor
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> The current elasticsearch templates do not support any logs other than Conn,
> HTTP, and DNS. We should provide additional templates so that an
> out-of-the-box bro install can send all of its logs into Metron and they will
> get probably indexed in elasticsearch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
