Daily selenium tests have been successfully migrated off of betacluster and onto catalyst, and this morning the jobs on betacluster have been deleted.
Daily tests ran in parallel on betacluster and catalyst and results were measured daily over two different two week blocks. The high level stats are listed below, and you can click on the phab links for the full rundown, which includes breakdown by suite and further specificities. 2026-08-13 -> 2026-08-26 36.2 point failure reduction Catalyst 98.1% eventually green vs betacluster 61.9% over the same 10 suites. Full rundown: https://phabricator.wikimedia.org/T435222#12259705 2026-09-01 -> 2026-09-14 16.6 point failure reduction. The difference from the first two week block is mostly betacluster improving (61.9% -> 79.1%) rather than catalyst regressing (98.1% -> 95.7%) Catalyst 95.7% eventually green vs betacluster 79.1% over the same 10 suites. Full rundown: https://phabricator.wikimedia.org/T435222#12319752 One note, no catalyst build went red because of a test failure - all 9 across both windows were environment provisioning, and a retry for that is already being investigated (T437955). 6 of those 9 were a single incident on Sept 4. As always, for your bookmarking pleasure, daily test results are reported here: https://www.mediawiki.org/wiki/Wikimedia_Quality_Services#Automated_tests_available
_______________________________________________ Wikitech-l mailing list -- [email protected] To unsubscribe send an email to [email protected] https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
