[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is WARNING **

Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: WARNING Date/Time: Thu 15 Mar 03:02:29 UTC 2018 Notes URLs: Additional Info: WARNING: 80.00% of data above the warning threshold [1.0]

[Betacluster-alerts] ** PROBLEM alert - Graphite Labs/Mediawiki Error Rate is CRITICAL **

Notification Type: PROBLEM Service: Mediawiki Error Rate Host: Graphite Labs Address: graphite-labs.wikimedia.org State: CRITICAL Date/Time: Thu 15 Mar 02:52:29 UTC 2018 Notes URLs: Additional Info: CRITICAL: 20.00% of data above the critical threshold [10.0]

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Free space - all mounts is CRITICAL **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: CRITICAL Date/Time: Thu 15 Mar 01:24:39 UTC 2018 Notes URLs: Additional Info: CRITICAL: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints

[Betacluster-alerts] ** PROBLEM alert - deployment-puppetmaster02/Long lived cherry-picks on puppetmaster is CRITICAL **

Notification Type: PROBLEM Service: Long lived cherry-picks on puppetmaster Host: deployment-puppetmaster02 Address: 10.68.21.200 State: CRITICAL Date/Time: Thu 15 Mar 00:31:16 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] ** RECOVERY alert - deployment-maps01/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-maps01 Address: 10.68.16.73 State: OK Date/Time: Thu 15 Mar 00:11:17 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] ** RECOVERY alert - deployment-maps01/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-maps01 Address: 10.68.16.73 State: OK Date/Time: Wed 14 Mar 23:42:13 UTC 2018 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-maps01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-maps01 Address: 10.68.16.73 State: CRITICAL Date/Time: Wed 14 Mar 23:36:16 UTC 2018 Notes URLs: Additional Info: CRITICAL: 33.33% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-maps01/Free space - all mounts is CRITICAL **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-maps01 Address: 10.68.16.73 State: CRITICAL Date/Time: Wed 14 Mar 23:27:14 UTC 2018 Notes URLs: Additional Info: CRITICAL: deployment-prep.deployment-maps01.diskspace.root.byte_percentfree (<22.22%)

[Betacluster-alerts] beta-scap-eqiad - Build # 199564 - Fixed!

beta-scap-eqiad - Build # 199564 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199564/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki05/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-mediawiki05 Address: 10.68.22.21 State: OK Date/Time: Wed 14 Mar 20:57:46 UTC 2018 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki05/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-mediawiki05 Address: 10.68.22.21 State: WARNING Date/Time: Wed 14 Mar 20:52:46 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-mediawiki05.diskspace.root.byte_percentfree (<11.11%)

[Betacluster-alerts] beta-scap-eqiad - Build # 199563 - Still Failing!

beta-scap-eqiad - Build # 199563 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199563/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki07/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-mediawiki07 Address: 10.68.17.40 State: CRITICAL Date/Time: Wed 14 Mar 20:39:00 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0]

[Betacluster-alerts] beta-scap-eqiad - Build # 199562 - Still Failing!

beta-scap-eqiad - Build # 199562 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199562/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 199561 - Still Failing!

beta-scap-eqiad - Build # 199561 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199561/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 199560 - Still Failing!

beta-scap-eqiad - Build # 199560 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199560/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 199559 - Still Failing!

beta-scap-eqiad - Build # 199559 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199559/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 199558 - Still Failing!

beta-scap-eqiad - Build # 199558 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199558/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 199557 - Failure!

beta-scap-eqiad - Build # 199557 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199557/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-mx/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-mx Address: 10.68.17.78 State: CRITICAL Date/Time: Wed 14 Mar 18:39:21 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-jobrunner02/Puppet errors is OK **

Notification Type: RECOVERY Service: Puppet errors Host: deployment-jobrunner02 Address: 10.68.19.42 State: OK Date/Time: Wed 14 Mar 17:28:40 UTC 2018 Notes URLs: Additional Info: OK: Less than 1.00% above the threshold [0.0] ___ Betacluster-alerts

[Betacluster-alerts] beta-scap-eqiad - Build # 199544 - Fixed!

beta-scap-eqiad - Build # 199544 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199544/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** PROBLEM alert - deployment-jobrunner02/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-jobrunner02 Address: 10.68.19.42 State: CRITICAL Date/Time: Wed 14 Mar 16:48:39 UTC 2018 Notes URLs: Additional Info: CRITICAL: 20.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] beta-scap-eqiad - Build # 199543 - Still Failing!

beta-scap-eqiad - Build # 199543 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199543/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 199542 - Still Failing!

beta-scap-eqiad - Build # 199542 - Still Failing: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199542/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 199541 - Failure!

beta-scap-eqiad - Build # 199541 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199541/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-code-update-eqiad - Build # 197696 - Fixed!

beta-code-update-eqiad - Build # 197696 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/197696/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-code-update-eqiad - Build # 197693 - Failure!

beta-code-update-eqiad - Build # 197693 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/197693/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] beta-scap-eqiad - Build # 199538 - Fixed!

beta-scap-eqiad - Build # 199538 - Fixed: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199538/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-tin/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: OK Date/Time: Wed 14 Mar 15:43:40 UTC 2018 Notes URLs: Additional Info: OK: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints found)

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: WARNING Date/Time: Wed 14 Mar 15:38:39 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints

[Betacluster-alerts] ** PROBLEM alert - deployment-redis02/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-redis02 Address: 10.68.16.231 State: CRITICAL Date/Time: Wed 14 Mar 15:33:41 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-redis01/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-redis01 Address: 10.68.16.177 State: CRITICAL Date/Time: Wed 14 Mar 15:27:43 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** RECOVERY alert - deployment-tin/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: OK Date/Time: Wed 14 Mar 14:47:38 UTC 2018 Notes URLs: Additional Info: OK: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints found)

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Free space - all mounts is CRITICAL **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: CRITICAL Date/Time: Wed 14 Mar 14:37:40 UTC 2018 Notes URLs: Additional Info: CRITICAL: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-tin Address: 10.68.21.205 State: WARNING Date/Time: Wed 14 Mar 14:32:39 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-tin.diskspace._mnt.byte_percentfree (No valid datapoints

[Betacluster-alerts] beta-scap-eqiad - Build # 199537 - Failure!

beta-scap-eqiad - Build # 199537 - Failure: Check console output at https://integration.wikimedia.org/ci/job/beta-scap-eqiad/199537/ to view the results.___ Betacluster-alerts mailing list Betacluster-alerts@lists.wikimedia.org

[Betacluster-alerts] ** RECOVERY alert - deployment-mediawiki04/Free space - all mounts is OK **

Notification Type: RECOVERY Service: Free space - all mounts Host: deployment-mediawiki04 Address: 10.68.19.128 State: OK Date/Time: Wed 14 Mar 10:59:46 UTC 2018 Notes URLs: Additional Info: OK: All targets OK ___ Betacluster-alerts mailing list

[Betacluster-alerts] ** PROBLEM alert - deployment-mediawiki04/Free space - all mounts is WARNING **

Notification Type: PROBLEM Service: Free space - all mounts Host: deployment-mediawiki04 Address: 10.68.19.128 State: WARNING Date/Time: Wed 14 Mar 10:54:49 UTC 2018 Notes URLs: Additional Info: WARNING: deployment-prep.deployment-mediawiki04.diskspace.root.byte_percentfree (<10.00%)

[Betacluster-alerts] ** PROBLEM alert - deployment-tin/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-tin Address: 10.68.21.205 State: CRITICAL Date/Time: Wed 14 Mar 09:18:36 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___

[Betacluster-alerts] ** PROBLEM alert - deployment-mira/Puppet errors is CRITICAL **

Notification Type: PROBLEM Service: Puppet errors Host: deployment-mira Address: 10.68.20.135 State: CRITICAL Date/Time: Wed 14 Mar 09:14:03 UTC 2018 Notes URLs: Additional Info: CRITICAL: 100.00% of data above the critical threshold [0.0] ___