On 19 Mar 2010, at 11:44, Noah Slater wrote: > Yep, I checked both browsers out of habit more than anything. > > What about the other thread discussing an actual issue brought up by some of > these failures?
It was an issue in the test case, not core code. > Is it not worrying that they failed first then worked, or is this something > ignorable? The latter. > And if so, what can we do to fix the whole situation post-release? (answering anyway :) Move all JS tests to etap tests to factor out browser issues, caching, timing, etc and don't introduce any errors. Cheers Jan -- > > On 19 Mar 2010, at 15:53, Jan Lehnardt wrote: > >> Hi Noah, >> >> The test suite doesn't support Safari. While I agree that the >> first-failing-then-working tests in Firefox are concerning, I say it doesn't >> block a release. >> >> Cheers >> Jan >> -- >> >> >> On 19 Mar 2010, at 06:59, Noah Slater wrote: >> >>> Hey, >>> >>> I was going to call a vote on the release today, but I am being blocked by >>> test suite errors. >>> >>> Running in Safari: >>> >>> changes >>> >>> • Assertion 'should return matching rows' failed: expected '3', got '1' >>> >>> rev_stemming >>> >>> • Assertion 'should return a truncated revision list' failed: expected >>> '5', got '6' >>> >>> The first one fails consistently, with debugging and without. No additional >>> details provided by the test. >>> >>> The second one is quite mysterious, however, and needs to be fixed. >>> >>> I ran it again with the debugger, and IT PASSED. >>> >>> I ran it again as normal, and got: >>> >>> • Assertion failed: db.open("bar", {revs:true})._revisions.ids.length >>> == newLimit + 1 >>> • Assertion failed: docB2._conflicts[0] == docB1._rev) // We having >>> already updated bar before setting the limit, so it's still got // a long >>> rev history. compact to stem the revs. T(db.open("bar", >>> {revs:true})._revisions.ids.length == newLimit + 1 >>> >>> I ran it again with the debugger: >>> >>> • Assertion failed: false >>> • Assertion failed: false >>> >>> I ran the whole thing from scratch, and I got the first error, then the >>> second error message for the second test. >>> >>> Running on Firefox: >>> >>> changes >>> >>> • Assertion 'JSON.parse(lines[2]).id == "rusty", lines[2]' failed: >>> {"last_seq":9} >>> • Exception raised: >>> {"message":"JSON.parse","fileName":"http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11.0","lineNumber":167,"stack":"(false)@http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11.0:167\u000arun(11)@http://127.0.0.1:5984/_utils/script/couch_test_runner.js?0.11.0:83\u000a"} >>> compactsuccess304ms >>> >>> list_views >>> >>> • Exception raised: {} >>> >>> Both tests pass when run a second time. >>> >>> A few other people on IRC were able to reproduce some of the errors I have >>> described here. >>> >>> I now consider these tests suspicious too, and have no idea how serious the >>> problems are. >>> >>> Any help or guidance would be fantastic. >>> >>> We're so close to the release! >>> >> >