This looks like a good time to fork this conversation as this is a good problem to fix. How can we notify developers when they break other things in the stack and how? On 7 Mar 2014 05:36, "Bartosz Dziewoński" <matma....@gmail.com> wrote:
> (continued, about the browser testing) > > (tl;dr where are the tests and how do I know they fail?) > > So, we have some slick browser tests. Awesome! But what's not good is > that the tests run off-site, the results are not reported back to > gerrit nor Bugzilla (unless someone manually files a bug, usually > Chris) not IRC nor anywhere else, and are generally non-discoverable > until someone shouts at you for breaking them. (As Tim guessed, I did > not know about any failures until Jon told me.) > > In fact, I still have no idea what exactly the tests encompass (I've > heard about some browser tests for VE because I lurk a lot, never > heard of any for core) or where to find them or how to run them. > Either I'm slow or we have a serious documentation failure here. > > Can something be done about it? Can we have the results reported > somewhere visible – preferably to gerrit, as jenkins already reports > some post-merge checks there? Or maybe we can have automatically filed > bug reports if the build breaks? A bot reporting test status on > #wikimedia-dev? Anything? > > (I understand that the tests take too long to run them in the > pre-merge checks.) > > (Jon proposed reverting problematic changes outright, but to me that > seems like a bit of an overreaction – bugs in tests and false > positives happen, let's not make a huge fuss out of that.) > > (to be continued: about the deployment) > > _______________________________________________ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
