Hi, here are my thoughts about phpunit and selenium testing.
> The wiki under test is set up with a master database consisting of a single > objectcache table. > The entries of this table specify a test run identifier as primary key > and temporary resource identifiers as dependent fields. If I understand this correctly, this would not allow to test any wikis that are running on live sites, e.g. intranet wikis. While I agree that regression testing on live sites is not a good idea, I kind of like the notion that after setting up a wiki with all the extensions I like to have, I could do some sort of "everything up and running"-test. With the concept of using separate testing databases and resources, this would be possible without interference with the actual data and could even be done at intervals during, say, maintenance periods. > Setup of a test run requires the creation of the test > run temporary resources and a entry in the objectcache table. Are there already mechanisms for this? I haven't done too much work with the objectcache. This is where memcached data is stored, right? So how do I get the data that is needed? This question leads me to another one: How do I get the testind database and resources? As I see this, it should be part of the testing framework to be able to produce the set of data needed from a "normal" MW installation. The whole mechanism would actually be something like a backup, so we might look into any existing solutions for that. > When a request is sent to the wiki under test, very early in the request > processing (e.g., immediately after LocalSettings is processed) a hook is > called with the provided state information as an argument that accesses > the objectcache table. The extension function handling the hook switches > in the temporary resources and returns. Also, this hook might handle the reconfiguration of the wiki, if needed. So for example, testing the PagedTiffHandler requires uploading of tiff files to be enabled. However, there might be some security risks, since it is not directly obvious in the code which settings are changed. So the hook should only be called when $wgEnableSelenium = true. In addition, we could define a DontTouchThisSettings.php, which is called even after the hook and holds some settings that are holy to the admin of the wiki :) The question is, though, would this not become somewhat too complicated? > After the test run completes, the testing application cleans up the test run > by requesting the deletion of the temporary resources and the objectcache > table > entry associated with the test run. In some cases, tests will not change any data, e.g. testing dynamic skin elements in vector skin. Would it make sense not to tear down the testing environment in that case in order to save some time when testing repeatedly? I think, there is a conflict between performance and amount of data, but who wins? In general, it seems to me that we have some similarity with what is called wiki family on mediawiki.org. One could see multiple testing environments as a set of multiple wikis that share a common codebase [1]. Does anybody have experience with wiki families and the object cache rsp. memcached? I am not sure whether we can use the same codebase as parsertests. I'd rather think, parser tests are a special case of what we are sketching here. On the other hand, I don't think it is a good idea to have two separate approaches for very similar tasks in the code. Do you think it would be feasible to separate the preparation part from both parser tests and selenium tests and build both of them on a common ground? Best regards, Markus [1] http://www.mediawiki.org/wiki/Wiki_family#Scenario_2:_Quick_set-up _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
