+ qa mailing list On Fri, Oct 3, 2014 at 10:46 AM, Jon Robson <[email protected]> wrote: > Would it be useful to send an email first thing Wednesday morning > listing all the failing tests per project with links and how long they > have been failing for. > > I personally would find this sort of email useful to remind me to > check status of code about to be merged. > > Thoughts? > Jon > > > On Thu, Sep 25, 2014 at 5:16 PM, Chris McMahon <[email protected]> wrote: >> >> >> On Thu, Sep 25, 2014 at 4:43 PM, S Page <[email protected]> wrote: >>> >>> The great news is Echo and Flow browser tests are catching real bugs: >>> * unwanted change in the behavior of moderated topics >>> * a template bug only for anonymous users >>> >>> The problem is we don't notice these when merging code. We rely on someone >>> later figuring out that the redness in a Jenkins browser run is significant >>> and not beta instability, and that sometimes happens after deployment. >> >> >> So to begin with, in no case in the last couple of weeks has "redness" been >> caused by "beta instability". I have merged five patches to Echo and two to >> Flow this week in the service of a) maintenance to make the builds green >> when they should be b) refactoring for ease of analysis and maintenance and >> c) workarounds for weirdness in our front end code. >> >> Echo is essentially done for now, I think it is about as good as it is going >> to get. It should be reliable and maintainable at this point. If the Echo >> tests go red, then Something Is Wrong (although it might only be that the >> tests need updating for new UI.) Echo is also a rather small set of tests. >> >> I'm going to keep refactoring the Flow tests until they are in the same >> state. I'll be looking for review help from S and from Dan Duvall along the >> way. >> >>> So "ACTION ....then need to run the tests related to what they're >>> changing" is a work in progress. >>> >>> What steps can the team take to address this? >> >> >> My suggestions: >> >> * run the browser tests in a Vagrant env before you merge changes to master >> ** if this is inconvenient, engage Dan Duvall until it is convenient. We've >> already done this for VE and it is in process for MobileFrontend, which is a >> bit more complicated. >> * if the build goes red and you don't know why, ask me or Zeljko or Dan or >> post to the QA mail list >> * if you can't figure out how to test some path through the app, ask me or >> Zeljko or Dan or post to the QA mail list. >> * if you want something tested but you have to write features dammit and you >> have no time to write browser tests, check in a patch that contains some >> Given/When/Then plain English test steps and put me on the review for it, >> we'll sort it out. >> >> _______________________________________________ >> E2 mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/e2 >>
_______________________________________________ QA mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/qa
