On Sat, Nov 16, 2013 at 3:22 PM, Chris McMahon <[email protected]>wrote:
> > On Fri, Nov 15, 2013 at 12:28 PM, Jeff Hall <[email protected]> wrote: > >> I spent some time today looking at automated test failures via >> CloudBees/Jenkins (https://wmf.ci.cloudbees.com), and a pretty common >> theme of tests that fail inconsistently is "Watir::Wait::TimeoutError" >> issues. Here's an example of a recent failure that falls into this >> category: >> >> >> https://wmf.ci.cloudbees.com/job/browsertests-commons.wikimedia.beta.wmflabs.org-linux-chrome/463/testReport/(root)/UploadWizard/Navigate_to_Describe_page/ >> > > Yes, this is a legitimate timeout issue, unlike the UW test failure on Nov > 14, another of those maddening unexplained js errors from a ResourceLoader > call: > https://wmf.ci.cloudbees.com/job/browsertests-commons.wikimedia.beta.wmflabs.org-linux-chrome/461/ > As some may not know, visiting a URL with ?debug=1 (or setting the cookie) may report JS errors in the browser console. See https://www.mediawiki.org/wiki/ResourceLoader/Developing_with_ResourceLoader#Toggle_debug_mode How did you determine it's a JS error? Was it by: Click cloudbees job, navigate to failing test. Click Sauce Labs job URL: http://saucelabs.com/jobs/a31c0d53909e436cb6d31fb264933615 link. In [Commands] tab search for "error", find click to view 4 console messages ( 1 error ) Expand it and see [200 OK] GET http://bits.beta.wmflabs.org/commons.wikimedia.beta.wmflabs.org/load.php?debug=false&lang=en&modules=jquery.ui.button%2Ccore%2Cdatepicker%2Cdialog%2Cdraggable%2Cmouse%2Cposition%2Cprogressbar%2Cresizable%2Cselectable%2Cwidget&skin=vector&version=20130920T230134Z&* Uncaught SyntaxError: Unexpected end of input There's an explicit test for RL errors in features/step_definitions/create_account_steps.rb in qa/browsertests.git has a Then(/^page has no ResourceLoader errors$/) do I think that assertion should be performed when visiting every page. > I agree that explicit sleep() calls are bad design, > Are there any implicit sleep() calls in tests? When I watch WebDriver run tests, it sometimes seems to wait a long time before moving to the next step, it's not clear what it's doing. The few sleep calls I've seen in tests are just 1 or 2 seconds. (Is there a verbose mode for cucumber?) > but I like polling for conditions: > > when_present() polls until an element can be engaged > wait_until() polls until a condition returns true > wait_while() polls until a condition returns false > That's gold, Jerry. Gold! I added it to https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Running_and_writing_tests#Timeouts -- =S Page Features engineer
_______________________________________________ QA mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/qa
