(continued, about the browser testing) (tl;dr where are the tests and how do I know they fail?)
So, we have some slick browser tests. Awesome! But what's not good is that the tests run off-site, the results are not reported back to gerrit nor Bugzilla (unless someone manually files a bug, usually Chris) not IRC nor anywhere else, and are generally non-discoverable until someone shouts at you for breaking them. (As Tim guessed, I did not know about any failures until Jon told me.) In fact, I still have no idea what exactly the tests encompass (I've heard about some browser tests for VE because I lurk a lot, never heard of any for core) or where to find them or how to run them. Either I'm slow or we have a serious documentation failure here. Can something be done about it? Can we have the results reported somewhere visible – preferably to gerrit, as jenkins already reports some post-merge checks there? Or maybe we can have automatically filed bug reports if the build breaks? A bot reporting test status on #wikimedia-dev? Anything? (I understand that the tests take too long to run them in the pre-merge checks.) (Jon proposed reverting problematic changes outright, but to me that seems like a bit of an overreaction – bugs in tests and false positives happen, let's not make a huge fuss out of that.) (to be continued: about the deployment) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
