https://bugzilla.wikimedia.org/show_bug.cgi?id=72722
Bug ID: 72722
Summary: Jenkins tests shouldn't go red when it's not its fault
Product: Wikimedia
Version: wmf-deployment
Hardware: All
OS: All
Status: NEW
Severity: normal
Priority: Unprioritized
Component: Quality Assurance
Assignee: wikibugs-l@lists.wikimedia.org
Reporter: jrob...@wikimedia.org
CC: cmcma...@wikimedia.org, dduv...@wikimedia.org,
zfili...@wikimedia.org
Web browser: ---
Mobile Platform: ---
The MobileFrontend tests (in Firefox) have been flakey for some time. Take a
look at the list on the left for examples of recurring issues.
Example:
https://integration.wikimedia.org/ci/view/Mobile/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/333/
I propose that when a test fails and the content of the Stacktrace contains one
of
*Timeout::Error (Timeout::Error)
* Net::ReadTimeout
* 500 error (Our servers are currently experiencing a technical...)
the test should be marked as unreliable and should not be considered failure.
Could the test go amber when this happens instead? It would allow for better
visual scanning when things break.
If there are 2 timeout errors but one actual error the test should stay red as
there is a genuine error there.
--
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
_______________________________________________
Wikibugs-l mailing list
Wikibugs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
