On Thu, 05 Aug 2010 18:47:58 -0400, Mark A. Hershberger wrote: > Markus Glaser <gla...@hallowelt.biz> writes: > >> 1) Where are the tests located? I suggest for core to put them into >> maintenance/tests/selenium. That is where they are now. For extensions >> I propse a similar structure, that is <extensiondir>/tests/selenium. > > Sounds fine. > > In the same way, since maintenance/tests contains tests that should be > run using PHPUnit, we can say that <extensiondir>/tests will contain > tests that should be run using PHPUnit. > >> Alternatively, we could use the word "Selenium" somewhere in there in >> order to be able to divide between unit and selenium tests. > > I think putting them in the selenium directory (or the “Se” directory) > is sufficient. > >> 3) How does the framework know there are tests? > > Can I suggest that the framework can see that an extension has tests > simply by the presence of the <extensiondir>/tests directory containing > a <Extension>*TestSuite.php file? > > The <extensiondir>/tests/<Extension>TestSuite.php file should define a > class using the name <Extension>TestSuite which has a static method > suite(). See the PHPUnit documentation at http://bit.ly/b9L50r for how > this is set up. > > This static suite() method should take care of letting the autoloader > know about any test classes so the test classes are only available > during testing. > > So, for your example using PagedTiffHandler, there would be the files: > > PagedTiffHandler/tests/PagedTiffHandlerTestSuite.php > PagedTiffHandler/tests/PagedTiffHandlerUploadsTestSuite.php > >> 4) Which tests should be executed? > > By default all the test suites in <extensiondir>/tests should be run. > > It is should be possible to specify which particular test to run by > using whatever command line arguments to the CLI. > > This seems better to me than defining a new global. If some tests > should only be run rarely, that information can be put in the TestSuite > class for te extension. > > In this way, I think it is possible to remove all the $wgSelenium* > variables from the DefaultSettings.php file. > > (I plan to do something similar with the $wgParserTest* variables as > well — these sorts of things don't seem like they belong in Core.) > > Mark.
I don't think walking through all the extensions looking for test subdirectories and then running all tests therein is a good idea. First, in a large installation with many extensions, this takes time and delays the test execution. Second, if a developer is working on a particular extension or part of the core, it will be common to run the tests associated with that for regression purposes. Making the developer specify the extension or core tests to run on the RunSeleniumTests command line is irritating (at least, it would irritate me). Specifying the test suite(s) to be run in LocalSettings.php is a set and forget approach that allows the developer to get on with the work. However, I do agree that the number of global variables associated with the selenium framework is getting large and has the potential of growing over time. One solution is to use a multi-dimensional associative array (much like $wgGroupPermissions). We could use a global variable $wgSelenium and move all selenium framework values into it. For example: $wgSelenium['wiki']['host'] = 'localhost'; $wgSelenium['wiki']['wikiurl'] = false; $wgSelenium['wiki']['loginname'] = 'Wikiuser; $wgSelenium['wiki']['password'] = ''; $wgSelenium['server']['port'] = 4444; etc. The only global we may wish to keep separate is $wgEnableSelenium, since it specifies whether $wgSelenium is used. -- -- Dan Nessett _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
