Re: [Wikitech-l] QUnit testing in Jenkins
On Tue, Mar 5, 2013 at 4:12 AM, Krinkle krinklem...@gmail.com wrote: As of today, we automatically run our QUnit test suite[4] in MediaWiki core from Jenkins. Great news! I won't go in detail about what PhantomJS is, but in short: It is a headless WebKit browser. Meaning, it doesn't render pixels to a screen on the server side, but it does behave like a fully valid browser environment as if it were rendering it to a screen (CSS is being parsed, the DOM is there, stylesheets are active, retrieving computed styles, ajax requests can be made etc.). For more information, see [2]. PhantomJS can render pixels, but I am not sure if it does it by default, or only when requested. The reason I know it can render pixels is that you can take a screen shot while driving PhantomJS with Selenium. Screenshots from a headless browser. Think about it for a minute. :) Željko ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] QUnit testing in Jenkins
On Tue, Mar 5, 2013 at 4:12 AM, Krinkle krinklem...@gmail.com wrote: However, unlike php-checkstyle, our QUnit tests are actually passing From console[1]: 02:50:21 Testing http://localhost:9412/mediawiki-core-28a705a9f648da310ff2a4fca9d013bf147f3d1f/index.php?title=Special:JavaScriptTest/qunitExceptionthrown by test.module1: expected 02:50:21 Error: expected 02:50:24 OK 02:50:24 832 assertions passed (5350ms) 02:50:24 02:50:24 Done, without errors. This is strange. The console says there was a problem: Exception thrown by test.module1: expected and Error: expected But then it says everything is fine: Done, without errors. Željko -- [1] https://integration.mediawiki.org/ci/job/mediawiki-core-qunit/3/console ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] QUnit testing in Jenkins
From console[1]: 02:50:21 Testing http://localhost:9412/mediawiki-core-28a705a9f648da310ff2a4fca9d013bf147f3d1f/index.php?title=Special:JavaScriptTest/qunitExceptionthrown by test.module1: expected 02:50:21 Error: expected 02:50:24 OK 02:50:24 832 assertions passed (5350ms) 02:50:24 02:50:24 Done, without errors. This is strange. The console says there was a problem: Exception thrown by test.module1: expected and Error: expected But then it says everything is fine: Done, without errors. Željko -- [1] https://integration.mediawiki.org/ci/job/mediawiki-core-qunit/3/console I'm not sure how to navigate to the source that defines that test but I suspect it's just using an expected exception test: http://docs.jquery.com/QUnit/raises ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] QUnit testing in Jenkins
On Mar 5, 2013, at 3:49 PM, Dan Andreescu dandree...@wikimedia.org wrote: From console[1]: 02:50:21 Testing http://localhost:9412/mediawiki-core-28a705a9f648da310ff2a4fca9d013bf147f3d1f/index.php?title=Special:JavaScriptTest/qunitExceptionthrown by test.module1: expected 02:50:21 Error: expected 02:50:24 OK 02:50:24 832 assertions passed (5350ms) 02:50:24 02:50:24 Done, without errors. This is strange. The console says there was a problem: Exception thrown by test.module1: expected and Error: expected But then it says everything is fine: Done, without errors. Željko -- [1] https://integration.mediawiki.org/ci/job/mediawiki-core-qunit/3/console I'm not sure how to navigate to the source that defines that test but I suspect it's just using an expected exception test: http://docs.jquery.com/QUnit/raises No, it is neither. Remember you're looking at the console log. Which, in this case is being written two from lots of sources: * jenkins - stdout * grunt/qunit - stdout * phantomjs - console.log() The part cited here is mostly qunit's output (the dotted progress line), but when calling console.log from within the javascript it is forwarded to the jenkins console. A console log is harmless and no reason for alarm. If it were an actual exception, it wouldn't be tolerated. In this case it is coming from the unit tests that asserts that mw.loader sets a module's state to error if it's javascript bundle throws an exception. mw.loader executes the bundle in a try/catch and logs any exceptions to the console after which it sets state=error and triggers the error callbacks. Right now grunt/qunit forwards the phantomjs console straight to the main output. It would be prettier if it would display it in a way that more clearly says phantomjs console. I requested this 3 months ago: https://github.com/gruntjs/grunt-contrib-qunit/pull/6 Execute the tests in your own browser and you'll see the same data in your browser's console (e.g. Chrome Dev Tools). -- Krinkle ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] QUnit testing in Jenkins
On Mon, Mar 4, 2013 at 8:07 PM, Ori Livneh o...@wikimedia.org wrote: Today I sprinted to pick up QUnit testing in Jenkins and get it stabilised and deployed. This is fantastic. Thanks, Timo. Indeed - this is a great milestone. Thanks for all your work getting this out the door, Timo! :-) Erik -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] QUnit testing in Jenkins
On 03/04/2013 07:12 PM, Krinkle wrote: Things this will catch are basically everything else. Any runtime error that we can't detect in static analysis but will fail no matter what browser you're in, such as: * misspelled identifiers or syntax errors * issues with ResourceLoader (mw.loader) * issues with AJAX * any code failures that result in exceptions * the obvious (catching failures/regressions in our QUnit tests) This is really a great step forward. Thanks, Matt Flaschen ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] QUnit testing in Jenkins
On Mar 4, 2013 7:13 PM, Krinkle krinklem...@gmail.com wrote: Hey all, As of today, we automatically run our QUnit test suite[4] in MediaWiki core from Jenkins. Awesome, thanks Krinkle! Greg ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] QUnit testing in Jenkins
On Monday, March 4, 2013 at 7:12 PM, Krinkle wrote: Today I sprinted to pick up QUnit testing in Jenkins and get it stabilised and deployed. This is fantastic. Thanks, Timo. -- Ori Livneh ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] QUnit testing in Jenkins
Le 04/03/13 19:12, Krinkle wrote: As of today, we automatically run our QUnit test suite[4] in MediaWiki core from Jenkins. Congratulations Timo on this huge improvement to the continuous integration system \O/ -- Antoine hashar Musso ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
