[Betacluster-alerts] ** RECOVERY alert - deployment-ores-redis/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-ores-redis Address: 10.68.21.235 State: OK Date/Time: Thu 01 Sep 04:30:12 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-ms-fe01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-ms-fe01 Address: 10.68.16.96 State: OK Date/Time: Thu 01 Sep 04:11:42 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-ores-redis/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-ores-redis Address: 10.68.21.235 State: CRITICAL Date/Time: Thu 01 Sep 03:55:13 UTC 2016 Additional Info: CRITICAL: 50.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-ms-fe01/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-ms-fe01 Address: 10.68.16.96 State: CRITICAL Date/Time: Thu 01 Sep 03:36:43 UTC 2016 Additional Info: CRITICAL: 55.56% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-ms-fe01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-ms-fe01 Address: 10.68.16.96 State: OK Date/Time: Thu 01 Sep 01:40:42 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki02/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-mediawiki02 Address: 10.68.16.127 State: OK Date/Time: Thu 01 Sep 01:27:15 UTC 2016 Additional Info: OK: deployment-prep.deployment-mediawiki02.diskspace._srv.byte_percentfree (No valid datapoints found)

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki02/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-mediawiki02 Address: 10.68.16.127 State: WARNING Date/Time: Thu 01 Sep 01:22:15 UTC 2016 Additional Info: WARNING: deployment-prep.deployment-mediawiki02.diskspace._srv.byte_percentfree (No valid datapoints

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki02/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-mediawiki02 Address: 10.68.16.127 State: WARNING Date/Time: Wed 31 Aug 23:56:15 UTC 2016 Additional Info: WARNING: deployment-prep.deployment-mediawiki02.diskspace._srv.byte_percentfree (No valid datapoints

[Betacluster-alerts] ** RECOVERY alert - deployment-salt02/Puppet staleness is OK **

Notification Type: RECOVERY Service: Puppet staleness Host: deployment-salt02 Address: 10.68.17.58 State: OK Date/Time: Wed 31 Aug 23:51:27 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [3600.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-redis01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-redis01 Address: 10.68.16.177 State: OK Date/Time: Wed 31 Aug 20:45:54 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-redis01/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-redis01 Address: 10.68.16.177 State: CRITICAL Date/Time: Wed 31 Aug 20:05:51 UTC 2016 Additional Info: CRITICAL: 40.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-elastic07/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-elastic07 Address: 10.68.17.187 State: OK Date/Time: Wed 31 Aug 18:16:00 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-zookeeper01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-zookeeper01 Address: 10.68.17.157 State: OK Date/Time: Wed 31 Aug 15:59:49 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-fluorine02/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-fluorine02 Address: 10.68.23.106 State: OK Date/Time: Wed 31 Aug 15:59:49 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka05/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-kafka05 Address: 10.68.21.106 State: OK Date/Time: Wed 31 Aug 15:59:36 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-elastic08/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-elastic08 Address: 10.68.17.188 State: OK Date/Time: Wed 31 Aug 15:54:39 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-poolcounter02/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-poolcounter02 Address: 10.68.23.77 State: OK Date/Time: Wed 31 Aug 15:54:39 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-cache-text04/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-cache-text04 Address: 10.68.18.103 State: OK Date/Time: Wed 31 Aug 15:54:27 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-restbase01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-restbase01 Address: 10.68.16.128 State: OK Date/Time: Wed 31 Aug 15:49:29 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-mediawiki01 Address: 10.68.17.170 State: OK Date/Time: Wed 31 Aug 15:49:10 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka04/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-kafka04 Address: 10.68.17.9 State: CRITICAL Date/Time: Wed 31 Aug 15:24:48 UTC 2016 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-poolcounter02/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-poolcounter02 Address: 10.68.23.77 State: CRITICAL Date/Time: Wed 31 Aug 15:24:39 UTC 2016 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mathoid/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-mathoid Address: 10.68.23.236 State: CRITICAL Date/Time: Wed 31 Aug 15:24:41 UTC 2016 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-logstash2/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-logstash2 Address: 10.68.16.147 State: CRITICAL Date/Time: Wed 31 Aug 15:24:51 UTC 2016 Additional Info: CRITICAL: 87.50% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka05/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-kafka05 Address: 10.68.21.106 State: CRITICAL Date/Time: Wed 31 Aug 15:24:37 UTC 2016 Additional Info: CRITICAL: 50.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-fluorine02/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-fluorine02 Address: 10.68.23.106 State: CRITICAL Date/Time: Wed 31 Aug 15:24:48 UTC 2016 Additional Info: CRITICAL: 87.50% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-parsoid09/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-parsoid09 Address: 10.68.20.142 State: CRITICAL Date/Time: Wed 31 Aug 15:24:35 UTC 2016 Additional Info: CRITICAL: 87.50% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-restbase01/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-restbase01 Address: 10.68.16.128 State: CRITICAL Date/Time: Wed 31 Aug 15:24:29 UTC 2016 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-cache-text04/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-cache-text04 Address: 10.68.18.103 State: CRITICAL Date/Time: Wed 31 Aug 15:24:27 UTC 2016 Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Mobile Main page is CRITICAL **

Notification Type: PROBLEM Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Wed 31 Aug 15:12:25 UTC 2016 Additional Info: CRITICAL - Socket timeout after 10 seconds

[Betacluster-alerts] ** PROBLEM alert - Generic Beta Cluster/English Wikipedia Main page is CRITICAL **

Notification Type: PROBLEM Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: CRITICAL Date/Time: Wed 31 Aug 15:11:17 UTC 2016 Additional Info: CRITICAL - Socket timeout after 10 seconds ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki02/App Server Main HTTP Response is CRITICAL **

Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki02 Address: 10.68.16.127 State: CRITICAL Date/Time: Wed 31 Aug 15:12:42 UTC 2016 Additional Info: CRITICAL - Socket timeout after 10 seconds ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki03/App Server Main HTTP Response is CRITICAL **

Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki03 Address: 10.68.17.55 State: CRITICAL Date/Time: Wed 31 Aug 15:14:29 UTC 2016 Additional Info: CRITICAL - Socket timeout after 10 seconds ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki01/App Server Main HTTP Response is CRITICAL **

Notification Type: PROBLEM Service: App Server Main HTTP Response Host: deployment-mediawiki01 Address: 10.68.17.170 State: CRITICAL Date/Time: Wed 31 Aug 15:13:11 UTC 2016 Additional Info: CRITICAL - Socket timeout after 10 seconds ___

[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Main page is OK **

Notification Type: RECOVERY Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Wed 31 Aug 15:21:07 UTC 2016 Additional Info: HTTP OK: HTTP/1.1 200 OK - 45600 bytes in 1.320 second response time

[Betacluster-alerts] ** RECOVERY alert - deployment-pdf01/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-pdf01 Address: 10.68.16.73 State: OK Date/Time: Wed 31 Aug 15:20:47 UTC 2016 Additional Info: OK: deployment-prep.deployment-pdf01.diskspace.root.byte_percentfree (More than half of the datapoints are undefined)

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki03/App Server Main HTTP Response is OK **

Notification Type: RECOVERY Service: App Server Main HTTP Response Host: deployment-mediawiki03 Address: 10.68.17.55 State: OK Date/Time: Wed 31 Aug 15:19:21 UTC 2016 Additional Info: HTTP OK: HTTP/1.1 200 OK - 45168 bytes in 1.471 second response time

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki01/App Server Main HTTP Response is OK **

Notification Type: RECOVERY Service: App Server Main HTTP Response Host: deployment-mediawiki01 Address: 10.68.17.170 State: OK Date/Time: Wed 31 Aug 15:18:01 UTC 2016 Additional Info: HTTP OK: HTTP/1.1 200 OK - 45160 bytes in 1.225 second response time

[Betacluster-alerts] beta-code-update-eqiad - Build # 119437 - Failure!

beta-code-update-eqiad - Build # 119437 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/119437/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Mobile Main page is OK **

Notification Type: RECOVERY Service: English Wikipedia Mobile Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Wed 31 Aug 15:06:17 UTC 2016 Additional Info: HTTP OK: HTTP/1.1 200 OK - 33186 bytes in 1.348 second response time

[Betacluster-alerts] ** RECOVERY alert - Generic Beta Cluster/English Wikipedia Main page is OK **

Notification Type: RECOVERY Service: English Wikipedia Main page Host: Generic Beta Cluster Address: en.wikipedia.beta.wmflabs.org State: OK Date/Time: Wed 31 Aug 15:05:10 UTC 2016 Additional Info: HTTP OK: HTTP/1.1 200 OK - 45600 bytes in 1.321 second response time

[Betacluster-alerts] ** RECOVERY alert - deployment-kafka05/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-kafka05 Address: 10.68.21.106 State: OK Date/Time: Wed 31 Aug 13:33:36 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-kafka05/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-kafka05 Address: 10.68.21.106 State: CRITICAL Date/Time: Wed 31 Aug 12:53:35 UTC 2016 Additional Info: CRITICAL: 33.33% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-aqs01/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-aqs01 Address: 10.68.18.237 State: OK Date/Time: Wed 31 Aug 09:47:47 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-aqs01/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-aqs01 Address: 10.68.18.237 State: CRITICAL Date/Time: Wed 31 Aug 09:37:45 UTC 2016 Additional Info: CRITICAL: 44.44% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** PROBLEM alert - deployment-redis01/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-redis01 Address: 10.68.16.177 State: CRITICAL Date/Time: Wed 31 Aug 09:03:52 UTC 2016 Additional Info: CRITICAL: 20.00% of data above the critical threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki02/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-mediawiki02 Address: 10.68.16.127 State: OK Date/Time: Wed 31 Aug 08:59:14 UTC 2016 Additional Info: OK: deployment-prep.deployment-mediawiki02.diskspace._srv.byte_percentfree (No valid datapoints found)

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki02/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-mediawiki02 Address: 10.68.16.127 State: WARNING Date/Time: Wed 31 Aug 08:49:17 UTC 2016 Additional Info: WARNING: deployment-prep.deployment-mediawiki02.diskspace._srv.byte_percentfree (No valid datapoints

[Betacluster-alerts] Host DOWN alert for deployment-parsoid05!

Notification Type: PROBLEM Host: deployment-parsoid05 State: DOWN Address: 10.68.16.120 Info: CRITICAL - Host Unreachable (10.68.16.120) Date/Time: Wed 31 Aug 07:50:25 UTC 2016 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-mx/Puppet run is OK **

Notification Type: RECOVERY Service: Puppet run Host: deployment-mx Address: 10.68.17.78 State: OK Date/Time: Wed 31 Aug 07:33:44 UTC 2016 Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts mailing list

[Betacluster-alerts] Host UP alert for deployment-parsoid05!

Notification Type: RECOVERY Host: deployment-parsoid05 State: UP Address: 10.68.16.120 Info: PING OK - Packet loss = 0%, RTA = 0.49 ms Date/Time: Wed 31 Aug 07:10:09 UTC 2016 ___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-mx/Puppet run is CRITICAL **

Notification Type: PROBLEM Service: Puppet run Host: deployment-mx Address: 10.68.17.78 State: CRITICAL Date/Time: Wed 31 Aug 06:53:42 UTC 2016 Additional Info: CRITICAL: 30.00% of data above the critical threshold [0.0] ___ Betacluster-alerts