[Wikidata-bugs] [Maniphest] [Commented On] T144948: some icinga checks on WDQS do not send notifications

2016-09-15 Thread Gehel 
Gehel added a comment.
So it seems that puppet failures on wdqs1001 are notified on IRC (# wikidata), but 'WDQS HTTP Port' are not. Looking at the check definition on neon:/etc/icinga/puppet_services.cfg I don't see a significant difference:

define service {
# --PUPPET_NAME-- wdqs1001 WDQS_Internal_HTTP_endpoint
active_checks_enabled  1
check_command  nrpe_check!check_WDQS_Internal_HTTP_endpoint!10
check_freshness0
check_period   24x7
contact_groups admins,wdqs-admins
host_name  wdqs1001
is_volatile0
max_check_attempts 3
normal_check_interval  1
notification_interval  0
notification_options   c,r,f
notification_period24x7
passive_checks_enabled 1
retry_check_interval   1
servicegroups  wdqs_eqiad
service_descriptionWDQS HTTP Port

}



define service {
# --PUPPET_NAME-- wdqs1001 puppet_checkpuppetrun
active_checks_enabled  1
check_command  nrpe_check!check_puppet_checkpuppetrun!10
check_freshness0
check_period   24x7
contact_groups admins,wdqs-admins
host_name  wdqs1001
is_volatile0
max_check_attempts 3
normal_check_interval  1
notification_interval  0
notification_options   c,r,f
notification_period24x7
passive_checks_enabled 1
retry_check_interval   1
servicegroups  wdqs_eqiad
service_descriptionpuppet last run

}

Looking at neon:/var/log/icinga/irc-wikidata.log it seems that notifications were sent for both of those checks:

gehel@neon:/var/log/icinga$ grep -i puppet irc-wikidata.log | tail -n 2
PROBLEM - puppet last run on wdqs1001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues  
RECOVERY - puppet last run on wdqs1001 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures  
gehel@neon:/var/log/icinga$ grep -i 'WDQS HTTP Port' irc-wikidata.log | tail -n 2
PROBLEM - WDQS HTTP Port on wdqs1002 is CRITICAL: Connection refused by host  
RECOVERY - WDQS HTTP Port on wdqs1002 is OK: TCP OK - 0.000 second response time on 127.0.0.1 port 80TASK DETAILhttps://phabricator.wikimedia.org/T144948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: GehelCc: Smalyshev, Gehel, Aklapper, hoo, mschwarzer, Avner, debt, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, aude, Deskana, Manybubbles, faidon, Mbch331, Jay8g___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T144948: some icinga checks on WDQS do not send notifications

2016-09-15 Thread Gehel 
Gehel added a comment.
Actually, it seems that only the WDQS_Lag check is not reported to the wdqs-admins group. Patch coming up.TASK DETAILhttps://phabricator.wikimedia.org/T144948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: GehelCc: Smalyshev, Gehel, Aklapper, hoo, mschwarzer, Avner, debt, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, aude, Deskana, Manybubbles, faidon, Mbch331, Jay8g___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T144948: some icinga checks on WDQS do not send notifications

2016-09-15 Thread gerritbot 
gerritbot added a comment.
Change 310748 had a related patch set uploaded (by Gehel):
wdqs - send notifications of WDQS lag also to wdqs-admins group

https://gerrit.wikimedia.org/r/310748TASK DETAILhttps://phabricator.wikimedia.org/T144948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Gehel, gerritbotCc: gerritbot, Smalyshev, Gehel, Aklapper, hoo, mschwarzer, Avner, debt, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, aude, Deskana, Manybubbles, faidon, Mbch331, Jay8g___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T144948: some icinga checks on WDQS do not send notifications

2016-09-15 Thread gerritbot 
gerritbot added a comment.
Change 310748 merged by Gehel:
wdqs - send notifications of WDQS lag also to wdqs-admins group

https://gerrit.wikimedia.org/r/310748TASK DETAILhttps://phabricator.wikimedia.org/T144948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Gehel, gerritbotCc: gerritbot, Smalyshev, Gehel, Aklapper, hoo, mschwarzer, Avner, Lewizho99, Maathavan, debt, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas, aude, Deskana, Manybubbles, faidon, Mbch331, Jay8g___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs