tl;dr below, frustration ahead.
I've been trying to get Jenkins to run the parsertests of
LabeledSectionTransclusion, but this has been an incredibly frustrating
experience. It shouldn't be too hard, right? Well, read on...
I thought I'd start with the existing 'testextensions' jenkins job [1],
which is failing. The reason it is failing is because of this:
*12:02:56* + php tests/phpunit/phpunit.php --log-junit
junit-phpunit-ext.xml --
extensions/LabeledSectionTransclusion*12:02:56* Unexpected
non-MediaWiki exception encountered, of type
"PHPUnit_Framework_Exception"*12:02:56* exception
'PHPUnit_Framework_Exception' with message 'Neither
"extensions/LabeledSectionTransclusion.php" nor
"extensions/LabeledSectionTransclusion.php" could be opened.' in
/usr/share/php/PHPUnit/Util/Skeleton/Test.php:100
After installing PHPUnit (on itself quite a frustrating experience), I
get this response instead:
valhallasw@lisilwen:~/src/core$ php tests/phpunit/phpunit.php
extensions/LabeledSectionTransclusion
PHPUnit 3.7.10 by Sebastian Bergmann.
Configuration read from /home/valhallasw/src/core/tests/phpunit/suite.xml
Time: 0 seconds, Memory: 18.50Mb
No tests executed!
Which suggests it might be a problem with the Jenkins configuration.
Of course, the tests are also not running, but this is a second issue.
So, where is the Jenkins configuration? Not in Jenkins itself. After
logging in in Jenkins, there /is/ suddenly a new option 'Workspace'
(why is that disabled for logged-out users?), which shows the
extension is checked out in the workspace - so that doesn't explain
why phpunit is unable to locate the tests.
So let's check the configuration. [2] suggests it should be at [3].
Ugh, gitweb, and no github mirror. Under 'jobs', there are some
extensions, but not LST. There is MediaWiki-Tests-Extensions, which
has some XML files that, at least for me, are incomprehensible. And
are they even relevant to this? In any case, they don't tell me why
PHPUnit is unable to find the LST directory, even though it's right
there [4]. I give up.
By the way - I just realized why the job is not in the Jenkins repository:
valhallasw@lisilwen:~/src/jenkins$ grep -R * -e 'mwext'
jobs/.gitignore:/mwext-*
*Really?*
Okay, so then how do I get PHPUnit to run the parsertests? There is
the NewParserTest class, so I was guessing adding this to
LabeledSectionTransclusion.php would work:
class LSTParserTest extends NewParserTest {
function __construct()
{
parent::__construct();
$this->setParserTestFile(__DIR__ . "/lstParserTests.txt");
}
};
Well, that does, er, absolutely nothing. [5] suggests I should move it
to a seperate file and add the following:
$wgHooks['UnitTestsList'][] = 'efFruitsRegisterUnitTests';function
efFruitsRegisterUnitTests( &$files ) {
$testDir = dirname( __FILE__ ) . '/';
$files[] = $testDir . 'AppleTest.php';
return true;}
what it *doesn't* tell you is that this is actually irrelevant when
you run php tests/phpunit/phpunit.php
extensions/LabeledSectionTransclusion - and it's not really clear to
me for when it is relevant. After some struggling, apparently it
detects tests because the filename *ends in Test.php* (which is
documented, well, no-where. After moving the test case there, it still
didn't quite work, but after some more fiddling, something is working!
Okay, so now I have a test case that actually runs when I call the
above command. So, let's push it for review [6] - but that doesn't
actually make Jenkins run the test cases.... so I still don't know
whether this actually fixes it.
[1]
https://integration.mediawiki.org/ci/job/mwext-LabeledSectionTransclusion-testextensions/
[2] http://www.mediawiki.org/wiki/Continuous_integration/Jenkins
[3] https://gerrit.wikimedia.org/r/gitweb?p=integration/jenkins.git;a=tree
[4]
https://integration.mediawiki.org/ci/job/mwext-LabeledSectionTransclusion-testextensions/ws/extensions/
[5]
http://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Writing_unit_tests_for_extensions
So, tl,dr: debugging jenkins failures and getting phpunit to work is a
frustrating experience, because of a lack of documentation and a lack of
'obviousness' (there is not one 'obvious' way to do it). I've updated [5]
with my experiences, but I still have no idea how the magic that is Jenkins
is configured and how it works. It would be great if there would be better
documentation on that.
Some specific things that I think should be fixed:
- Jenkins should not show new (read-only) options to users - make them
public instead
- The Jenkins configuration should also be mirrored on github
- The mwext jobs should be in git (preferrably, because then it's easy to
check them on-line), *or* there should be clear documentation on how to
generate those jobs...
- There should be documentation on how the configuration is actually used
- and preferrably on how to debug failures.
Thanks.
Merlijn
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
