Filippo Giunchedi has submitted this change and it was merged.
Change subject: Only return mostly fresh data for elasticsearch ganglia
monitoring
......................................................................
Only return mostly fresh data for elasticsearch ganglia monitoring
The current stats in ganglia don't look entirely accurate. To take an example
ganglia reports that elastic1004 has had a steady 20M es_docs_count over the
last couple months, but the internal api ganglia queries currently reports 79M
docs. Similarly for elastic1024 ganglia reports around 20M es_docs_count, and
the api reports 38M.
I'm not certain that this will fix that, but it at least removes one question
about if somehow stale data is being served.
Change-Id: I85cd720f556cae417b99b2bcdde63c727133b530
---
M modules/elasticsearch/files/ganglia/elasticsearch_monitoring.py
1 file changed, 9 insertions(+), 0 deletions(-)
Approvals:
Filippo Giunchedi: Verified; Looks good to me, approved
diff --git a/modules/elasticsearch/files/ganglia/elasticsearch_monitoring.py
b/modules/elasticsearch/files/ganglia/elasticsearch_monitoring.py
index 01134d1..7e4ca42 100644
--- a/modules/elasticsearch/files/ganglia/elasticsearch_monitoring.py
+++ b/modules/elasticsearch/files/ganglia/elasticsearch_monitoring.py
@@ -28,6 +28,9 @@
d.update(stat)
return d
+# Maximum time to server stale stats
+LAST_FETCH_THRESHOLD = 600
+
# Stat types
GAUGE = {
'slope': 'both',
@@ -477,6 +480,12 @@
if data['stats'] is None:
return None
+ # Don't keep returning stale data forever
+ now = time.time()
+ diff = now - data['last_fetch']
+ if diff > LAST_FETCH_THRESHOLD:
+ return None
+
path = data['path_transformer'](data, stats[name]['path'])
val = dig_it_up(data['stats'], path)
--
To view, visit https://gerrit.wikimedia.org/r/212322
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I85cd720f556cae417b99b2bcdde63c727133b530
Gerrit-PatchSet: 4
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: EBernhardson <ebernhard...@wikimedia.org>
Gerrit-Reviewer: EBernhardson <ebernhard...@wikimedia.org>
Gerrit-Reviewer: Filippo Giunchedi <fgiunch...@wikimedia.org>
Gerrit-Reviewer: Giuseppe Lavagetto <glavage...@wikimedia.org>
Gerrit-Reviewer: Manybubbles <never...@wikimedia.org>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
