I enjoyed the session we did in San Francisco on Cucumber and Page Objects, and wondered about other subject that might be of interest.
One suggestion I have is a workshop on analyzing why tests fail. At any given time we almost always have failing builds at https://wmf.ci.cloudbees.com . (Right now we have a lot of failing builds, some of them are because of real bugs, some are not.) I think it would be interesting to have a session where we look at the error messages in Jenkins that our automation tools produce; relate that information to the code in that test; relate the code to what the browser did in the test; and then tell a story about what made the test fail. What do you think? So for instance when you look at https://wmf.ci.cloudbees.com/job/browsertests-en.wikipedia.beta.wmflabs.org-linux-firefox/142/testReport/%28root%29/ULS%20cog%20behaviour%20on%20the%20side-bar%20for%20logged%20in%20users/Page_without_interlanguage_links_has_cog_icon_and_language_selector_on_Talk_page/ and you see something like unable to locate element, using {:class=>"uls-settings-trigger", :tag_name=>"span"} (Watir::Exception::UnknownObjectException) what does that message mean? What happened to cause that failure? In this case the failure is because of this: https://bugzilla.wikimedia.org/show_bug.cgi?id=50624 Having testers looking at these build failures and discussing them here would be really useful to WMF, but it is also a subject of general interest I think.
_______________________________________________ QA mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/qa
