On Sat, Feb 6, 2010 at 16:27, Chad <innocentkil...@gmail.com> wrote: > On Sat, Feb 6, 2010 at 11:24 AM, Ævar Arnfjörð Bjarmason > <ava...@gmail.com> wrote: >> On Sat, Feb 6, 2010 at 01:04, Ævar Arnfjörð Bjarmason <ava...@gmail.com> >> wrote: >>> Since the tests were ported from t/ to phpunit's >>> phase3/maintenance/tests/ in r61938 and other commits running the >>> tests on my machine takes up to 1GB of memory and grows as it runs >>> more tests. It seems that phpunit uses the same instance of the php >>> interpreter for running all the tests. >>> >>> Is there some way around this? Perhaps phpunit.xml could be tweaked so >>> that it runs a new php for each test? >>> >>> Furthermore when I run `make test' I get this: >>> >>> Time: 03:35, Memory: 1849.25Mb >>> >>> There were 2 failures: >>> >>> 1) LanguageConverterTest::testGetPreferredVariantUserOption >>> Failed asserting that two strings are equal. >>> --- Expected >>> +++ Actual >>> @@ @@ >>> -tg-latn >>> +tg >>> >>> >>> /home/avar/src/mw/trunk/phase3/maintenance/tests/LanguageConverterTest.php:82 >>> >>> 2) Warning >>> No tests found in class "ParserUnitTest". >>> >>> FAILURES! >>> Tests: 686, Assertions: 3431, Failures: 2, Incomplete: 34 >>> >>> But when I run phpunit manually on the test then all tests pass: >>> >>> $ phpunit LanguageConverterTest.php >>> PHPUnit 3.4.5 by Sebastian Bergmann. >>> >>> ......... >>> >>> Time: 23 seconds, Memory: 23.75Mb >>> >>> OK (9 tests, 34 assertions) >>> >>> Also after I get "Tests: 686, Assertions: 3431, Failures: 2, >>> Incomplete: 34" in the first output phpunit doesn't exit and continues >>> hugging my memory. Why is it still running? It has already run all the >>> tests. >> >> I've worked around this by adding a 'make tap' target which runs the >> phpunit tests individually with Test::Harness. I made it the default >> target due to the problems with running all the tests at once with >> phpunit: >> >> http://www.mediawiki.org/wiki/Special:Code/MediaWiki/62071 >> http://www.mediawiki.org/wiki/Special:Code/MediaWiki/62072 >> >> Does something run these tests or the parsertests automatically? It >> would be really neat to test all svn revisions of MediaWiki and report >> the results on Special:Code. I think I read somewhere that something >> runs the parsertests automatically. > > It's supposed to be running the parser tests and uploading them on > commit, but that's been broken for a little while now.
What system is this that's running automatic tests on commits? I was investigating setting up a buildbot (http://buildbot.net/) which could have multiple test clients and report tests to IRC/XML which Special:Code could then use. What does the now-broken Special:Code test system use? > If we're got a > nice standard output from the tests (I think the XML is pretty suited > for this), we should be able to upload that result to Code Review. $ prove -j 10 -e 'phpunit --tap' -Q *Test*.php All tests successful. Files=20, Tests=692, 31 wallclock secs ( 0.34 usr 0.21 sys + 18443939634.30 cusr 2803481.20 csys = 18446743116.05 CPU) Result: PASS You can get pretty HTML like this: $ prove --formatter TAP::Formatter::HTML -j 10 -e 'phpunit --tap' -Q *Test*.php > ~/www/mw-tap-out.html Which gives you something like this: http://v.nix.is/~avar/mw-tap-out.html That can be parsed with any XML parser that just has to look for <td class="results"> and <div id="summary" class="passed"> or <div id="summary" class="failed"> _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
