EBernhardson has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/344044 )
Change subject: Update mwgrep for elasticsearch 5.x
......................................................................
Update mwgrep for elasticsearch 5.x
* the 'filtered' query has been removed, use a straight bool+filter
* We don't care about result order, so sort by _doc which basically
skips the sorting step.
* Update cluster settings to allow querting > 1k shards. This limit
exist to protect the cluster from bad queries, but this is our
expected behaviour. It's unfortunate it has to be global instead of
specified in the query. This has been applied to transient cluster
settings already, the setting here ensures on restart it is picked
back up.
Bug: T161055
Change-Id: I58ee998b5edd7914dd44a9acc0f15e92e7987b66
---
M modules/elasticsearch/templates/elasticsearch_5.yml.erb
M modules/scap/files/mwgrep
2 files changed, 8 insertions(+), 1 deletion(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/44/344044/1
diff --git a/modules/elasticsearch/templates/elasticsearch_5.yml.erb
b/modules/elasticsearch/templates/elasticsearch_5.yml.erb
index 87aea61..b3accc1 100644
--- a/modules/elasticsearch/templates/elasticsearch_5.yml.erb
+++ b/modules/elasticsearch/templates/elasticsearch_5.yml.erb
@@ -360,6 +360,12 @@
action.destructive_requires_name: true
##
+# Allow up to 5k shards to be queried at a time. Increased from the default
+# 1k to allow mwgrep to operate
+##
+action.search.shard_count.limit: 5000
+
+##
# Enable the disk space aware shard allocator
##
cluster.routing.allocation.disk.threshold_enabled: true
diff --git a/modules/scap/files/mwgrep b/modules/scap/files/mwgrep
index 0b4ce44..886c680 100755
--- a/modules/scap/files/mwgrep
+++ b/modules/scap/files/mwgrep
@@ -117,7 +117,8 @@
search = {
'size': args.max_results,
'_source': ['namespace', 'title'],
- 'query': {'filtered': {'filter': {'bool': {'must': filters}}}},
+ 'sort': ['_doc'],
+ 'query': {'bool': {'filter': filters}},
'stats': ['mwgrep'],
}
--
To view, visit https://gerrit.wikimedia.org/r/344044
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I58ee998b5edd7914dd44a9acc0f15e92e7987b66
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: EBernhardson <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
