BryanDavis has uploaded a new change for review. https://gerrit.wikimedia.org/r/298295
Change subject: logstash: Update default mappings for Elasticsearch 2.x ...................................................................... logstash: Update default mappings for Elasticsearch 2.x Update the default mapping template to coerce all non-string fields without explicit mappings to strings. Starting with Elasticsearch 2.x, fields with the same name, in the same index, in different types, must have the same mapping. This is problematic for our Logstash traffic where different applications may (and do!) use common names like "code" and "status" as structured log data with differing content. We have a "normalize_fields" filter that has been used to try and clean up these differences, but that is a fragile approach that could be broken at any time by a new application or new event type for an existing application that has conflicts with anything else in our logging environment. Bug: T136001 Change-Id: I638d88e1d874fdb8be211bd74a1e36998d42dc09 --- M files/logstash/elasticsearch-template.json 1 file changed, 17 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/95/298295/1 diff --git a/files/logstash/elasticsearch-template.json b/files/logstash/elasticsearch-template.json index 3974ecb..54e791b 100644 --- a/files/logstash/elasticsearch-template.json +++ b/files/logstash/elasticsearch-template.json @@ -20,6 +20,8 @@ "enabled" : true, "omit_norms": true }, + "date_detection" : false, + "numeric_detection": false, "dynamic_templates" : [ { "string_fields" : { "match" : "*", @@ -39,6 +41,13 @@ "doc_values": true } } + } + }, + "force_to_string" : { + "match_mapping_type" : "*", + "mapping" : { + "type" : "string", + "index" : "not_analyzed" } } } ], @@ -73,6 +82,10 @@ "index" : "not_analyzed", "doc_values": true }, + "line" : { + "type" : "long", + "doc_values": true + }, "message" : { "type" : "string", "index" : "analyzed", @@ -93,6 +106,10 @@ } } }, + "pid" : { + "type" : "long", + "doc_values": true + }, "sequence_id" : { "type": "long", "doc_values": true -- To view, visit https://gerrit.wikimedia.org/r/298295 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I638d88e1d874fdb8be211bd74a1e36998d42dc09 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: BryanDavis <bda...@wikimedia.org> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits