BryanDavis has uploaded a new change for review.
https://gerrit.wikimedia.org/r/298295
Change subject: logstash: Update default mappings for Elasticsearch 2.x
......................................................................
logstash: Update default mappings for Elasticsearch 2.x
Update the default mapping template to coerce all non-string fields
without explicit mappings to strings.
Starting with Elasticsearch 2.x, fields with the same name, in the same
index, in different types, must have the same mapping. This is
problematic for our Logstash traffic where different applications may
(and do!) use common names like "code" and "status" as structured log
data with differing content. We have a "normalize_fields" filter that
has been used to try and clean up these differences, but that is
a fragile approach that could be broken at any time by a new application
or new event type for an existing application that has conflicts with
anything else in our logging environment.
Bug: T136001
Change-Id: I638d88e1d874fdb8be211bd74a1e36998d42dc09
---
M files/logstash/elasticsearch-template.json
1 file changed, 17 insertions(+), 0 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/95/298295/1
diff --git a/files/logstash/elasticsearch-template.json
b/files/logstash/elasticsearch-template.json
index 3974ecb..54e791b 100644
--- a/files/logstash/elasticsearch-template.json
+++ b/files/logstash/elasticsearch-template.json
@@ -20,6 +20,8 @@
"enabled" : true,
"omit_norms": true
},
+ "date_detection" : false,
+ "numeric_detection": false,
"dynamic_templates" : [ {
"string_fields" : {
"match" : "*",
@@ -39,6 +41,13 @@
"doc_values": true
}
}
+ }
+ },
+ "force_to_string" : {
+ "match_mapping_type" : "*",
+ "mapping" : {
+ "type" : "string",
+ "index" : "not_analyzed"
}
}
} ],
@@ -73,6 +82,10 @@
"index" : "not_analyzed",
"doc_values": true
},
+ "line" : {
+ "type" : "long",
+ "doc_values": true
+ },
"message" : {
"type" : "string",
"index" : "analyzed",
@@ -93,6 +106,10 @@
}
}
},
+ "pid" : {
+ "type" : "long",
+ "doc_values": true
+ },
"sequence_id" : {
"type": "long",
"doc_values": true
--
To view, visit https://gerrit.wikimedia.org/r/298295
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I638d88e1d874fdb8be211bd74a1e36998d42dc09
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: BryanDavis <bda...@wikimedia.org>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
