QChris has uploaded a new change for review. https://gerrit.wikimedia.org/r/186091
Change subject: Bump alert thresholds for EventLogging's overall events/s ...................................................................... Bump alert thresholds for EventLogging's overall events/s Since EventLogging volume recently outgrew the 350 events/s and EventLogging is known to be able to handle more events/s, we bump the threshold to avoid getting unneeded Icinga warnings for EventLogging. The 450 events/s threshold is arbitrary, but EventLogging is known to currently handle 450 events/s amount of traffic, and that threshold will silence the false alarms for now. The Analytics team has to come up with more realistic thresholds (T86244). Change-Id: I2312d62fff1ad851640c2f2fced646478833b7a4 --- M modules/eventlogging/manifests/monitoring.pp 1 file changed, 6 insertions(+), 3 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/91/186091/1 diff --git a/modules/eventlogging/manifests/monitoring.pp b/modules/eventlogging/manifests/monitoring.pp index 97c064b..3695f70 100644 --- a/modules/eventlogging/manifests/monitoring.pp +++ b/modules/eventlogging/manifests/monitoring.pp @@ -41,13 +41,16 @@ # of incoming events. class eventlogging::monitoring::graphite { - # Alarms if 1% of overall event throughput goes beyond 350 req/sec + # Warn if 1% of overall event throughput goes beyond 450 events/s # in a 15 min period + # These thresholds are somewhat arbtirary at this point, but it + # was seen that the current setup can handle 450 events/s. + # Better thresholds are pending (see T86244). monitoring::graphite_threshold { 'eventlogging_throughput': description => 'Throughput of event logging events', metric => 'eventlogging.overall.raw.rate', - warning => 350, - critical => 500, + warning => 450, + critical => 600, from => '15min', contact_group => 'analytics' } -- To view, visit https://gerrit.wikimedia.org/r/186091 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2312d62fff1ad851640c2f2fced646478833b7a4 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: QChris <christ...@quelltextlich.at> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits