Re: [Wikitech-l] Instructions for setting up regression tests on local machine?
On Fri, 30 Dec 2011 20:11:30 +, Dan Nessett wrote: > I have poked around a bit (using Google), but have not found > instructions for setting up the MW regression test framework (e.g., > CruiseControl or Jenkins or whatever is now being used + PHPUnit tests + > Selenium tests) on a local machine (so new code can be regression tested > before submitting patches to Bugzilla). Do such instructions exist and > if so, would someone provide a pointer to them? > > Thanks, The regression tests are failing badly on my local development machine. I am trying to figure out why and have proposed one possibility (https:// bugzilla.wikimedia.org/show_bug.cgi?id=33663). If anyone with the necessary understanding of the test suite would comment, that would help chasing down the problem. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] postgreSQL testing
On Thu, 12 Jan 2012 16:17:00 +0100, Antoine Musso wrote: > Hello, > > I have added a new continuous integration job to check our postgres > support. > This is exactly the same job as MediaWiki-phpunit, only the database > backend > change. > > The link is: >https://integration.mediawiki.org/ci/job/MediaWiki-postgres-phpunit/ > > As of now, there are two tests failing. I ran the JS tests for the following configuration and got 8 failures. How should I report the problems (e.g., dump the html into a file and attach it to a bug report?) Configuration: OS: CentOS 5.7 MW revision: 108821 PHP: 5.3.3 PHPUnit: 3.6.7 DB: Postgres 8.3.9 -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] postgreSQL testing
On Thu, 12 Jan 2012 17:59:09 +, Dan Nessett wrote: > On Thu, 12 Jan 2012 16:17:00 +0100, Antoine Musso wrote: > >> Hello, >> >> I have added a new continuous integration job to check our postgres >> support. >> This is exactly the same job as MediaWiki-phpunit, only the database >> backend >> change. >> >> The link is: >>https://integration.mediawiki.org/ci/job/MediaWiki-postgres-phpunit/ >> >> As of now, there are two tests failing. > > While I am grateful for the inclusion of a postgres backend in the > integration tests, I just ran make safe on MW 108734 and got 1 error, 25 > failures and 12 incomplete tests. Any idea why the local run has > different results that the automated run? > > I have attached the most recent run output to bug 33663. Sorry, my mistake. I forgot to run update. After doing so, I also get only 2 failures. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] postgreSQL testing
On Thu, 12 Jan 2012 16:17:00 +0100, Antoine Musso wrote: > Hello, > > I have added a new continuous integration job to check our postgres > support. > This is exactly the same job as MediaWiki-phpunit, only the database > backend > change. > > The link is: >https://integration.mediawiki.org/ci/job/MediaWiki-postgres-phpunit/ > > As of now, there are two tests failing. While I am grateful for the inclusion of a postgres backend in the integration tests, I just ran make safe on MW 108734 and got 1 error, 25 failures and 12 incomplete tests. Any idea why the local run has different results that the automated run? I have attached the most recent run output to bug 33663. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] postgreSQL testing
On Thu, 12 Jan 2012 16:17:00 +0100, Antoine Musso wrote: > Hello, > > I have added a new continuous integration job to check our postgres > support. > This is exactly the same job as MediaWiki-phpunit, only the database > backend > change. > > The link is: >https://integration.mediawiki.org/ci/job/MediaWiki-postgres-phpunit/ > > As of now, there are two tests failing. Excellent. Thank you for this. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Instructions for setting up regression tests on local machine?
On Wed, 11 Jan 2012 11:17:33 +0100, Antoine Musso wrote: > Le Wed, 11 Jan 2012 02:31:47 +0100, Dan Nessett a > écrit: >> Sure, I can post the results, but I don't think I should just dump them >> into this list (there are over 700 lines of output). How would you like >> me to go about it? > > You could open a bug report on https://bugzilla.wikimedia.org/ and > attach the output. > Or feel free to send it to Platonides and me, I can create the bug > report for you. I have created a bug report (https://bugzilla.wikimedia.org/show_bug.cgi? id=33663) and attached the output to it. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Instructions for setting up regression tests on local machine?
On Tue, 10 Jan 2012 23:53:25 +0100, Platonides wrote: > On 10/01/12 19:52, Dan Nessett wrote: >> I gave up on Ubuntu 8.04 and moved to Centos 5.7. After getting make >> safe to work, I get 27 failures and 14 incomplete tests. This is for >> revision 108474. Is there any way to know if this is expected? For >> example, are the results of nightly runs posted anywhere? > > It's not. > > Although it may be a configuration issue (eg. our script is not setting > $wgEnableUploads = true, but is assuming it is) > > Can you post the results? Sure, I can post the results, but I don't think I should just dump them into this list (there are over 700 lines of output). How would you like me to go about it? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Instructions for setting up regression tests on local machine?
On Fri, 30 Dec 2011 20:11:30 +, Dan Nessett wrote: > I have poked around a bit (using Google), but have not found > instructions for setting up the MW regression test framework (e.g., > CruiseControl or Jenkins or whatever is now being used + PHPUnit tests + > Selenium tests) on a local machine (so new code can be regression tested > before submitting patches to Bugzilla). Do such instructions exist and > if so, would someone provide a pointer to them? > > Thanks, I gave up on Ubuntu 8.04 and moved to Centos 5.7. After getting make safe to work, I get 27 failures and 14 incomplete tests. This is for revision 108474. Is there any way to know if this is expected? For example, are the results of nightly runs posted anywhere? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Instructions for setting up regression tests on local machine?
On Mon, 09 Jan 2012 09:26:24 +0100, Krinkle wrote:
> On Fri, Jan 6, 2012 at 8:06 PM, OQ wrote:
>
>> On Fri, Jan 6, 2012 at 12:56 PM, Dan Nessett
>> wrote:
>> > On Thu, 05 Jan 2012 14:03:14 -0600, OQ wrote:
>> >> uninstall the pear version and do a make install.
>> >
>> > Didn't work.
>> >
>> > # make install
>> > ./install-phpunit.sh
>> > Installing phpunit with pear
>> > Channel "pear.phpunit.de" is already initialized Adding Channel
>> > "components.ez.no" succeeded Discovery of channel "components.ez.no"
>> > succeeded Channel "pear.symfony-project.com" is already initialized
>> > Did not download optional dependencies: pear/Image_GraphViz,
>> > pear/Log, symfony/YAML, use --alldeps to download automatically
>> > phpunit/PHPUnit can optionally use package "pear/Image_GraphViz"
>> > (version
>> >>= 1.2.1)
>> > phpunit/PHPUnit can optionally use package "pear/Log" phpunit/PHPUnit
>> > can optionally use package "symfony/YAML" (version >= 1.0.2)
>> > downloading PHPUnit-3.4.15.tgz ...
>> > Starting to download PHPUnit-3.4.15.tgz (255,036 bytes)
>> > .done: 255,036
>> > bytes install ok: channel://pear.phpunit.de/PHPUnit-3.4.15
>>
>> Dunno then, it installed 3.6.3 for me. Hopefully somebody here knows a
>> bit more about pear :)
>>
>>
> I remember having a similar issue when I first installed phpunit on my
> Mac. Although I don't know the exact command, I remember having to
> update some central installer proces by PEAR ("channel" ?) to the latest
> version, which still had an old
> version in it's (local?) database.
>
> Krinkle
So this thread is complete, Ubuntu 8.04 is pinned to PHP 5.2.4 through
the standard distribution channels. PHPUnit 5.3 requires at least PHP
5.2.7. I looked around and could find no simple way to upgrade to PHP
5.2.x, x >= 7 on Ubuntu 8.04. I could have downloaded and tried to
upgrade manually, but frankly I didn't want to handle all of the version
incompatibily insanity that would entail.
For the record, here is an explanation how to update the pear channels
for PHPUnit 5.3 - http://sebastian-bergmann.de/archives/897-
PHPUnit-3.5.html
--
-- Dan Nessett
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Instructions for setting up regression tests on local machine?
On Thu, 05 Jan 2012 14:03:14 -0600, OQ wrote: > On Thu, Jan 5, 2012 at 1:34 PM, Dan Nessett wrote: >> So, I upgraded PHPUnit (using pear upgrade phpunit/PHPUnit). This >> installed 3.4.15 (not 3.5). >> >> I am running on Ubuntu 8.04. Anyone have an idea how to get PHPUnit 3.5 >> installed? >> >> > uninstall the pear version and do a make install. Didn't work. # make install ./install-phpunit.sh Installing phpunit with pear Channel "pear.phpunit.de" is already initialized Adding Channel "components.ez.no" succeeded Discovery of channel "components.ez.no" succeeded Channel "pear.symfony-project.com" is already initialized Did not download optional dependencies: pear/Image_GraphViz, pear/Log, symfony/YAML, use --alldeps to download automatically phpunit/PHPUnit can optionally use package "pear/Image_GraphViz" (version >= 1.2.1) phpunit/PHPUnit can optionally use package "pear/Log" phpunit/PHPUnit can optionally use package "symfony/YAML" (version >= 1.0.2) downloading PHPUnit-3.4.15.tgz ... Starting to download PHPUnit-3.4.15.tgz (255,036 bytes) .........done: 255,036 bytes install ok: channel://pear.phpunit.de/PHPUnit-3.4.15 -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Instructions for setting up regression tests on local machine?
On Fri, 30 Dec 2011 21:29:53 +0100, Roan Kattouw wrote: > On Fri, Dec 30, 2011 at 9:11 PM, Dan Nessett wrote: >> I have poked around a bit (using Google), but have not found >> instructions for setting up the MW regression test framework (e.g., >> CruiseControl or Jenkins or whatever is now being used + PHPUnit tests >> + Selenium tests) on a local machine (so new code can be regression >> tested before submitting patches to Bugzilla). Do such instructions >> exist and if so, would someone provide a pointer to them? >> > Jenkins is only really used to run the tests automatically when someone > commits. You ran run the PHPUnit tests locally without Jenkins. > Instructions on installing PHPUnit and running the tests is at > https://www.mediawiki.org/wiki/Manual:PHP_unit_testing . > > I don't have URLs for you offhand, but QUnit and Selenium are probably > also documented on mediawiki.org . > > Roan OK, I downloaded latest trunk and tried to run the PHPUnit tests. I executed make safe and got the following result: php phpunit.php --configuration /usr/local/src/mediawiki/latest_trunk/ trunk/phase3/tests/phpunit/suite.xml --exclude-group Broken,Destructive,Stub PHPUnit 3.5 or later required, you have 3.4.12 So, I upgraded PHPUnit (using pear upgrade phpunit/PHPUnit). This installed 3.4.15 (not 3.5). I am running on Ubuntu 8.04. Anyone have an idea how to get PHPUnit 3.5 installed? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Instructions for setting up regression tests on local machine?
I have poked around a bit (using Google), but have not found instructions for setting up the MW regression test framework (e.g., CruiseControl or Jenkins or whatever is now being used + PHPUnit tests + Selenium tests) on a local machine (so new code can be regression tested before submitting patches to Bugzilla). Do such instructions exist and if so, would someone provide a pointer to them? Thanks, -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Mediawiki 2.0
On Wed, 07 Dec 2011 12:54:22 +1100, Tim Starling wrote: > On 07/12/11 12:34, Dan Nessett wrote: >> On Wed, 07 Dec 2011 12:15:41 +1100, Tim Starling wrote: >>> How many servers do you have? >> >> 3. It would help to get it down to 2. >> >> I assume my comments apply to many other small wikis that use MW as >> well. Most operate on a shoe string budget. > > You should try running MediaWiki on HipHop. See > > http://www.mediawiki.org/wiki/HipHop > > It's not possible to pay developers to rewrite MediaWiki for less than > what it would cost to buy a server. But maybe getting a particular MW > installation to run on HipHop with a reduced feature set would be in the > same order of magnitude of cost. > > -- Tim Starling Are there any production wikis running MW over HipHop? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Mediawiki 2.0
On Wed, 07 Dec 2011 12:15:41 +1100, Tim Starling wrote: > On 07/12/11 09:50, Dan Nessett wrote: >> OK. Call it something else. The motivation for my question is getting >> server costs under control. Moving as much processing as possible >> client side is one way to achieve this. Cleaning up the security >> architecture may be overly ambitious, but a rewrite would provide the >> opportunity to take a rational look at MW's vulnerabilities and >> security services. >> >> I don't know where WMF is on the cost question, but we would benefit >> from reducing our hosting costs. > > WMF's hosting costs are pretty well under control. We have two parser > CPU reduction features on our roadmap, but the main justification for > doing them is to reduce latency, rather than cost, thus providing a > better user experience. If both of them are fully utilised, we can > probably reduce average parse time by a factor of 10. > > By "we" do you mean Citizendium? Yes. > How many servers do you have? 3. It would help to get it down to 2. I assume my comments apply to many other small wikis that use MW as well. Most operate on a shoe string budget. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Mediawiki 2.0
I assume the CC to me on the message below was a courtesy. However, I would like to comment on one point in Neil's message. I was not thinking about an application rewrite that allows projects "to jettison all existing wiki data." That would make no sense for us. We would need a way to convert the wiki data from the old format (i.e., mediawiki markup) to the new. Dan Nessett From: Neil Kandalgaonkar To: Wikimedia developers Cc: Dan Nessett Sent: Tuesday, December 6, 2011 4:18 PM Subject: Re: [Wikitech-l] Mediawiki 2.0 On 12/6/11 1:55 PM, Dan Nessett wrote: > This is a (admittedly long and elaborate) question, not a proposal. I ask > it in order to learn whether anyone has given it or something like it > some thought. > > Has anyone thought of creating MW 2.0? I mean by this, completely > rewriting the application in a way that may make it incompatible with MW > 1.x.y. In that case why not use some other wiki software? There are quite a few. See http://www.wikimatrix.org/>. If you're willing to jettison all existing wiki data I'm not sure I would recommend MediaWiki for a fresh start. I would in many cases, but not all. In any case, the WMF already have people working on a new parser, and a new GUI editor. If both of those projects reach fruition I would call that 2.0-worthy right there. Also, the new parser should provide us with some means to transition to a different wiki syntax if we think it's a good idea. If we wanted to *really* get radical, then we'd think about changing the storage model too, but you might as well rename it by that point. -- Neil Kandalgaonkar |) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Mediawiki 2.0
On Wed, 07 Dec 2011 09:26:50 +1100, Tim Starling wrote:
> On 07/12/11 08:55, Dan Nessett wrote:
>> This is a (admittedly long and elaborate) question, not a proposal. I
>> ask it in order to learn whether anyone has given it or something like
>> it some thought.
>>
>> Has anyone thought of creating MW 2.0? I mean by this, completely
>> rewriting the application in a way that may make it incompatible with
>> MW 1.x.y.
> [...]
>
>> * Get rid of mediawiki markup and move to html with embedded macros
>> that are processed client side.
>> * Move extension processing client side. * Replace templates with a
>> cleaner macro-based language (but, KISS).
>
> Keeping the same name ("MediaWiki") implies some level of compatibility
> with older versions of the same software. If you abandon existing
> installations and their needs altogether, then it makes sense to choose
> a new project name, so that the 1.x code can continue to be maintained
> and improved without causing user confusion.
>
> I think MediaWiki 2.0 should just be a renumbering, like Linux 2.6 ->
> 3.0, rather than any kind of backwards compatibility break.
>
> -- Tim Starling
OK. Call it something else. The motivation for my question is getting
server costs under control. Moving as much processing as possible client
side is one way to achieve this. Cleaning up the security architecture
may be overly ambitious, but a rewrite would provide the opportunity to
take a rational look at MW's vulnerabilities and security services.
I don't know where WMF is on the cost question, but we would benefit from
reducing our hosting costs.
--
-- Dan Nessett
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Mediawiki 2.0
On Wed, 07 Dec 2011 07:59:26 +1000, K. Peachey wrote: > http://www.mediawiki.org/wiki/Project:2.0 Thanks. I have moved my comments to that page's discussion. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Mediawiki 2.0
This is a (admittedly long and elaborate) question, not a proposal. I ask it in order to learn whether anyone has given it or something like it some thought. Has anyone thought of creating MW 2.0? I mean by this, completely rewriting the application in a way that may make it incompatible with MW 1.x.y. Pros * Improving the application architecture * Utilizing more client side resources, thereby reducing the server side resource requirements. * Clean up and improve existing services. Cons * Rewrites of major applications normally fail because they become overly ambitious. Some possible ways MW 2.0 might improve MW 1.x.y Change the parser - * Get rid of mediawiki markup and move to html with embedded macros that are processed client side. * Move extension processing client side. * Replace templates with a cleaner macro-based language (but, KISS). Pros * Reduce server side resource requirements, thereby reducing server side costs. Server side becomes mostly database manipulation. * Make use of the far larger aggregate resources available on client side (many more client machines than server machines). * With macro processing client side, debates about enhancements to parser extensions that require more processing shift to looking at client side. * Allows development of a parser driven by well-defined grammar. Cons * Unclear whether it is possible to move all or most parser processing to client side. * Would need a (probably large and complex) transition application that translates mediawiki markup into new grammar. * Since not all clients may have the resources to expand macros and do other client side processing in a timely manner, may need to provide server side surrogate processing based on either user selectable (e.g., preferences) choice or automatic discovery. * Need to select client side processing engine (e.g., Javascript, Java), which may lead to major developer fighting. Clean up security architecture -- * Support per page permissions, ala' Unix file system model. * Integrate authentication with emerging global services (e.g., OpenID) without use of extensions. * Move group membership definition out of LocalSettings into database table. Pros * Chance to think through security requirements and craft clean solution. * Offload most authentication processing and login data protection to service providers that more sharply focus on its requirements. * Some customers have expressed interest in per page permissions. Cons * Changing security architectures is a notoriously difficult objective. Most attempts lead to bloated solutions that never work in practice. * Some developers oppose per page permissions. * Would need to develop WMF standards that authentication providers must meet before accepting them for WMF project login. This is sufficient to illustrate the direction of my curiosity, but there are other things that MW 2.0 could do that might be discussed, such as: * Change the page history model. When page is flagged stable, subsequent page changes occur to new draft page. Provide link to draft page on stable page. * Think through how to support multiple db backends so application development doesn't continually break this support. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Rules for text in languages/messages/Messages***.php
I recently fixed a bug (14901) and attached some patches to the bug ticket. Included were some changes to languages/messages/MessagesEn.php. Following the directions given at http://www.mediawiki.org/wiki/ Localisation#Changing_existing_messages, I contacted #mediawiki-i18n and asked them for instructions how to proceed with the internationalization part of the work. After looking over my changes, they pointed out that my new message text violated some requirements, specifically: + I had used "/n" rather than literal newlines in the text + I had put newlines in front of some messages + I had put trailing whitespace in some messages Since I was (and am) unfamiliar with MW internationalization workflow and since this text is destined for email messages, not display on a wiki page, I was puzzled by these requirements. However, it was explained to me that internationalization occurs by display of the text on translate.net, which means the text must conform to web page display rules. Consequently, the problems specified above interfere with the translation work. I understand I need to fix these problems. However, before proceeding I wonder if there are other constraints that I must observe. I have read the material at http://www.mediawiki.org/wiki/Localisation, but did not find the above constraints mentioned there. Is there another place where they are specified? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Skin specific logos
On Fri, 04 Feb 2011 15:01:20 +0100, Krinkle wrote:
> Op 3 feb 2011, om 22:42 heeft Dan Nessett het volgende geschreven:
>
>> On Thu, 03 Feb 2011 21:19:58 +, Dan Nessett wrote:
>>
>>> Our site has 4 skins that display the logo - 3 standard and 1 site-
>>> specific. The site-specific skin uses rounded edges for the individual
>>> page area frames, while the standard skins use square edges. This
>>> means
>>> a logo with square edges looks fine for the standard skins, but not
>>> for
>>> the site-specific skin. A logo with rounded edges has the opposite
>>> characteristic.
>>>
>>> The programmer who designed the site-specific skin solved this problem
>>> with a hack. The absolute url to a different logo with rounded edges
>>> is
>>> hardwired into the skin code. Therefore, if we want to reorganize
>>> where
>>> we keep the site logos (which we have done once already), we have to
>>> modify the site-specific skin code.
>>>
>>> While it is possible that no one else has this problem, I would
>>> imagine
>>> there are skins out there that would look better if they were able to
>>> use a skin specific logo (e.g., using a different color scheme or a
>>> different font).
>>>
>>> My question is: has this issue been addressed before? If so, and there
>>> is a good solution, I would appreciate hearing of it.
>>>
>>> Regards,
>>
>> I need to correct a mistake I made in this post (sorry for replying to
>> my
>> own question). The site-specific skin keeps its skin specific logo in
>> the
>> skin directory and the skin code uses > >text('stylepath') ?>/> php $this->text('stylename') ?> to get to that directory. So, the url
>> is
>> not hardwired.
>>
>> However, we would like to keep all of the logos in one place, so I
>> think
>> the question is still pertinent.
>>
>> --
>> -- Dan Nessett
>>
>>
> You could upload a file to your wiki (say, at [[File:Wiki.png]]) and
> then put the
> full path to that in LocalSettings.php in $wgLogo.
>
> ie.
> $wgLogo = "/w/images/b/bc/Wiki.png";
>
> And then use that variable in the skins (The default skins use it
> already)
That's an interesting idea. But, probably the definition should be (e.g.):
$wgLogo = "$wgUploadPath/b/bc/Wiki.png";
--
-- Dan Nessett
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Skin specific logos
On Thu, 03 Feb 2011 13:52:30 -0800, Brion Vibber wrote: > On Thu, Feb 3, 2011 at 1:19 PM, Dan Nessett wrote: > >> Our site has 4 skins that display the logo - 3 standard and 1 site- >> specific. The site-specific skin uses rounded edges for the individual >> page area frames, while the standard skins use square edges. This means >> a logo with square edges looks fine for the standard skins, but not for >> the site-specific skin. A logo with rounded edges has the opposite >> characteristic. >> >> The programmer who designed the site-specific skin solved this problem >> with a hack. The absolute url to a different logo with rounded edges is >> hardwired into the skin code. Therefore, if we want to reorganize where >> we keep the site logos (which we have done once already), we have to >> modify the site-specific skin code. >> >> While it is possible that no one else has this problem, I would imagine >> there are skins out there that would look better if they were able to >> use a skin specific logo (e.g., using a different color scheme or a >> different font). >> >> My question is: has this issue been addressed before? If so, and there >> is a good solution, I would appreciate hearing of it. >> >> > A couple ideas off the top of my head: > > * You could use CSS to apply rounded corners with border-radius and its > -vendor-* variants. (May not work on all browsers, but requires no > upkeep other than double-checking that the rounded variant still looks > good. Doesn't help with related issues like an alternate color scheme > for the logo in different skins.) > * Your custom skin could use a custom configuration variable, say > $wgAwesomeSkinLogo. Have it use this instead of the default logo, and > make sure both settings get updated together. * You could use a fixed > alternate path which can be determined by modifying the string in > $wgLogo. Be sure to always store and update the second logo image > correctly. > * You could create a script that applies rounded corners or changes > colors in an existing image file and saves a new one, then find some way > to help automate your process of creating alternate logo images in the > above. > > -- brion Thanks. I think the second idea works best for us. It also suggests the use of a global $wgSkinLogos that points to a directory where all of the skin logos are kept. Any reason why this is a bad idea? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Skin specific logos
On Thu, 03 Feb 2011 21:19:58 +, Dan Nessett wrote:
> Our site has 4 skins that display the logo - 3 standard and 1 site-
> specific. The site-specific skin uses rounded edges for the individual
> page area frames, while the standard skins use square edges. This means
> a logo with square edges looks fine for the standard skins, but not for
> the site-specific skin. A logo with rounded edges has the opposite
> characteristic.
>
> The programmer who designed the site-specific skin solved this problem
> with a hack. The absolute url to a different logo with rounded edges is
> hardwired into the skin code. Therefore, if we want to reorganize where
> we keep the site logos (which we have done once already), we have to
> modify the site-specific skin code.
>
> While it is possible that no one else has this problem, I would imagine
> there are skins out there that would look better if they were able to
> use a skin specific logo (e.g., using a different color scheme or a
> different font).
>
> My question is: has this issue been addressed before? If so, and there
> is a good solution, I would appreciate hearing of it.
>
> Regards,
I need to correct a mistake I made in this post (sorry for replying to my
own question). The site-specific skin keeps its skin specific logo in the
skin directory and the skin code uses text('stylepath') ?>/text('stylename') ?> to get to that directory. So, the url is
not hardwired.
However, we would like to keep all of the logos in one place, so I think
the question is still pertinent.
--
-- Dan Nessett
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Skin specific logos
Our site has 4 skins that display the logo - 3 standard and 1 site- specific. The site-specific skin uses rounded edges for the individual page area frames, while the standard skins use square edges. This means a logo with square edges looks fine for the standard skins, but not for the site-specific skin. A logo with rounded edges has the opposite characteristic. The programmer who designed the site-specific skin solved this problem with a hack. The absolute url to a different logo with rounded edges is hardwired into the skin code. Therefore, if we want to reorganize where we keep the site logos (which we have done once already), we have to modify the site-specific skin code. While it is possible that no one else has this problem, I would imagine there are skins out there that would look better if they were able to use a skin specific logo (e.g., using a different color scheme or a different font). My question is: has this issue been addressed before? If so, and there is a good solution, I would appreciate hearing of it. Regards, -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Thu, 23 Sep 2010 20:13:23 -0700, Brion Vibber wrote: > On Thu, Sep 23, 2010 at 7:19 PM, Dan Nessett wrote: > >> I appreciate your recent help, so I am going to ignore the tone of your >> last message and focus on issues. While a test run can set up, use and >> then delete the temporary resources it needs (i.e., db, images >> directory, etc.), you really haven't answered the question I posed. If >> the test run ends abnormally, then it will not delete those resources. >> There has to be a way to garbage collect orphaned dbs, images >> directories and cache entries. >> >> > Any introductory Unix sysadmin handbook will include examples of shell > scripts to find old directories and remove them, etc. For that matter > you could simply delete *all* the databases and files on the test > machine every day before test runs start, and not spend even a second of > effort worrying about cleaning up individual runs. > > Since each test database is a fresh slate, there is no shared state > between runs -- there is *no* need to clean up immediately between runs > or between test sets. > > >> My personal view is we should start out simple (as you originally >> suggested) with a set of fixed URLs that are used serially by test >> runs. Implementing this is probably the easiest option and would allow >> us to get something up and running quickly. This approach doesn't >> require significant development, although it does require a way to >> control access to the URLs so test runs don't step on each other. > > > What you suggest is more difficult and harder to implement than creating > a fresh database for each test run, and gives no clear benefit in > exchange. > > Keep it simple by *not* implementing this idea of a fixed set of URLs > which must be locked and multiplexed. Creating a fresh database & > directory for each run does not require any additional development. It > does not require devising any access control. It does not require > devising a special way to clean up resources or restore state. > > -- brion I am authentically sorry that you feel obliged to couch your last 2 replies in an offensive manner. You have done some very good work on the Mediawiki code and deserve a great deal of credit for it. It is clear you do not understand what I am proposing (that is, what I proposed after I accepted your suggestion to keep things simple): + Every test run does create a fresh database (and fresh images directory) for each run. It does this by first dropping the database associated with the last run, recursively deleting the phase3 directory holding the code from the previous run, checking out the revision for the current run (or if this is judged too expensive, we could hold the revisions in tar files and untar them into the directory), adjusting things so that the wiki will work (e.g., recursively chmoding the image directory so it is writable), and installing a LocalSettings file so things like imagemagick, texvc, etc. are locatable and global variables set appropriately. All of this is done in the directory associated with the fixed URL. + Before each test suite of a regression test runs it prepares the wiki. For example, if a prerequisite for the suite is the availability of an image, it uploads it before starting. + The regression test can be guarded by writing a lock file in the images directory (which protects all code and data directories). When the regression test completes, the lock file can be removed and the next regression test started. If there is only one test driver application running, the lock file is unnecessary. If the test driver application crashes for some reason, a simple utility can sweep the directory structures associated with the URLs and remove everything. While this is only a sketch, it is obviously simpler than attempting to set up a test run using the wikipedia family scheme. The previous test run db, images directory, etc. are deleted at the beginning of the next run that uses the fixed URL and its associated directory space. There is no need to "Configure your DNS with a wildcard A record, and apache with a server alias (like ServerAlias *.yourdomain.net)", something that may not be possible for some sites. There is no need to dynamically edit LocalSettings to fix up the upload directory global variable. As for cleaning up between runs, this simply ensures the database server doesn't become clogged with extraneous databases, that directory space is used efficiently and that memcached doesn't hold useless data. While it may be possible to get by with cleaning up every 24 hours, that the fresh wiki installation process cleans up these resources by default means such a global sweep is completely unnecessary. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Thu, 23 Sep 2010 18:10:37 -0700, Brion Vibber wrote: > On Thu, Sep 23, 2010 at 4:03 PM, Dan Nessett wrote: > >> Thinking about this a bit, we seem to have come full circle. If we use >> a URL per regression test run, then we need to multiplex wiki >> resources. When you set up a wiki family, the resources are permanent. >> But, for a test run, you need to set them up, use them and then reclaim >> them. The resources are the db, the images directory, cache data, etc. >> >> > Computers can delete files as well as create them. Drop the database and > remove the uploads directory, now it's gone *poof magic*. > > Nothing has to be "multiplexed" or "locked": you create it before you > start using it, and if you don't need it you can remove it when you're > done with it. Nothing else will try to use it because nothing else has > the same test & test run ID -- other tests will use *their* dedicated > wikis. There is no overlap. > > This is trivial given the idea of "make one dedicated wiki for each test > run" and a basic understanding of how bulk MediaWiki installations > operate. If you're trying to help plan how to run automated regression > testing for MediaWiki *without* having done your basic research on how > the system works, I strongly recommend you stop and do that first before > continuing. > > -- brion I appreciate your recent help, so I am going to ignore the tone of your last message and focus on issues. While a test run can set up, use and then delete the temporary resources it needs (i.e., db, images directory, etc.), you really haven't answered the question I posed. If the test run ends abnormally, then it will not delete those resources. There has to be a way to garbage collect orphaned dbs, images directories and cache entries. I can think of a number of ways to do this. A cron job could periodically sweep the locations where those resources reside and reclaim orphans. This would require marking them with a lifetime after which they would be retired. Alternatively, each time a new test run begins, it could sweep for orphaned resources (again using a lifetime marker). In regards to locking, this is required for the fixed URL scheme I originally described, not for multiplexing a common code base over multiple wikis, which is what the wikipedia family mechanism implements. My personal view is we should start out simple (as you originally suggested) with a set of fixed URLs that are used serially by test runs. Implementing this is probably the easiest option and would allow us to get something up and running quickly. This approach doesn't require significant development, although it does require a way to control access to the URLs so test runs don't step on each other. Once we have some experience, we can identify any shortcomings of this approach and incrementally improve it. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Thu, 23 Sep 2010 15:50:48 -0700, Brion Vibber wrote: > On Thu, Sep 23, 2010 at 2:54 PM, Dan Nessett wrote: > >> On Thu, 23 Sep 2010 14:41:32 -0700, Brion Vibber wrote: >> > There's no need to have a fixed set of URLs; just as with Wikimedia's >> > public-hosted sites you can add individually-addressable wikis >> > dynamically at whim without touching any Apache configuration. URL >> > rewriting, or wildcard hostnames, or whatever lets you make as many >> > distinct URLs as you like, funnel them through a single web server >> > and a single codebase, but have them running with different >> > databases. -- brion >> >> Are there instructions somewhere that describe how to do this? >> >> > http://www.mediawiki.org/wiki/Manual:Wiki_family#Wikimedia_Method > > -- brion Thinking about this a bit, we seem to have come full circle. If we use a URL per regression test run, then we need to multiplex wiki resources. When you set up a wiki family, the resources are permanent. But, for a test run, you need to set them up, use them and then reclaim them. The resources are the db, the images directory, cache data, etc. So, either we must use the fixed URL scheme I mentioned previously, or we are back to resource switching (admittedly using the approach already in place for the wikipedia family). The test run can set up the resources and reclaim them, but we also need to handle test runs that fail in the middle (due to e.g. system OS crashes, power outages, etc.). The resources allocated for such runs will become orphans and some sort of garbage collection is required. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Thu, 23 Sep 2010 14:41:32 -0700, Brion Vibber wrote: > On Thu, Sep 23, 2010 at 2:31 PM, Dan Nessett wrote: > >> Not sure I get this. Here is what I understand would happen when a >> developer checks in a revision: >> >> + A script runs that manages the various regression tests run on the >> revision (e.g., parserTests, PHPUnit tests, the Selenium-based >> regression test). >> >> + The Selenium regression test needs a URL to work with. There are a >> fixed set of these defined in httpd.conf. >> >> > There's no need to have a fixed set of URLs; just as with Wikimedia's > public-hosted sites you can add individually-addressable wikis > dynamically at whim without touching any Apache configuration. URL > rewriting, or wildcard hostnames, or whatever lets you make as many > distinct URLs as you like, funnel them through a single web server and a > single codebase, but have them running with different databases. > -- brion Are there instructions somewhere that describe how to do this? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Thu, 23 Sep 2010 14:10:24 -0700, Brion Vibber wrote:
> + URLs identify test wikis. Only one regression test can run at time on
>> any one of these. How do you synchronize regression test initiation so
>> there is some sort of lock on a test wiki currently running a
>> regression test?
>>
>>
> Simplest way would be to have one wiki (database + URL) for each
> regression test ("test1234wiki"), or even for each run of each
> regression test ("test1234run432wiki").
>
> These could be created/removed as needed through simple shell scripting.
>
> -- brion
Not sure I get this. Here is what I understand would happen when a
developer checks in a revision:
+ A script runs that manages the various regression tests run on the
revision (e.g., parserTests, PHPUnit tests, the Selenium-based regression
test).
+ The Selenium regression test needs a URL to work with. There are a
fixed set of these defined in httpd.conf.
+ Before assigning one of these to the regression test run, there is a
requirement that it isn't currently busy running a regression test for a
different revision. So, you need resource access control on the URLs.
+ Once you have an idle URL, you can initialize the wiki per your
previous comments, including loading the revision into the directory
associated with the URL.
How does this fit into the idea of using a wiki per regression test or
regression test run?
--
-- Dan Nessett
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Thu, 23 Sep 2010 10:29:58 -0700, Brion Vibber wrote:
> On Thu, Sep 23, 2010 at 9:46 AM, Dan Nessett wrote:
>
>> I am very much in favor of keeping it simple. I think the issue is
>> whether we will support more than one regression test (or individual
>> test associated with a regression test) running concurrently on the
>> same test wiki. If not, then I agree, no switching logic is necessary.
>>
>>
> *nod*
>
> It might be a good idea to divide the test set up into, say 'channels'
> or 'bundles' which are independent of each other, but whose individual
> steps must run in sequence. If the tests are designed well, you should
> be able to run tests from multiple 'channels' on the same wiki
> simultaneously -- just as in the real world, multiple users are doing
> multiple things on your wiki at the same time.
>
> So one test set might be:
> * create page A-{{unique-id}} as user One-{{unique-id}} * open editing
> page as user One-{{unique-id}} * open and save the page as user
> Two-{{unique-id}} * save the page as user One-{{unique-id}} * confirm
> edit conflict / merging behavior was as expected
>
> And another might be:
> * register a new user account User--{{unique-id}} * change skin option
> in preferences
> * confirm that the skin changed as expected
>
> These tests don't interfere with each other -- indeed if they did, that
> would be information you'd need to know about a serious bug!
>
> Most test sets should be fairly separate like this; only some that
> change global state (say, a site administrator using a global
> configuration panel to change the default skin) would need to be run
> separately.
>
> -- brion
After thinking about this some more I think you are right. We should at
least start with something simple and only make it more complex (e.g.,
wiki resource switching) if the simple approach has significant problems.
There is already a way to 'bundle' individual tests together. It is the
selenium test suite. We could use that. We could then break up a
regression test into separate test suites that could run concurrently.
Summarizing, here is my understanding of your proposal:
+ A regression test run comprises a set of test suites, each of which may
run concurrently.
+ If you want to run multiple regression tests concurrently, use
different test wikis (which can run over the same code base, but which
are identified by different URLs - i.e., rely on httpd to multiplex
multiple concurrent regression tests).
+ If you want to run parts of a regression test concurrently, the unit of
concurrency is the test suite.
+ A regression test begins by establishing a fresh wiki. Each test suite
starts by establishing the wiki state it requires (e.g., for
PagedTiffHanlder, loading Multipage.tiff).
+ It is an open question whether test suite or total regression test
cleanup is necessary. It may be possible to elide this step and simply
rely on regression test initialization to cleanup any wiki state left
around by a previous test run.
There are still some open questions:
+ How do you establish a fresh wiki for a URL used previously for a test
run?
+ URLs identify test wikis. Only one regression test can run at time on
any one of these. How do you synchronize regression test initiation so
there is some sort of lock on a test wiki currently running a regression
test?
--
-- Dan Nessett
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Thu, 23 Sep 2010 09:24:18 -0700, Brion Vibber wrote: > Given a test matrix with multiple OSes, this ain't something individual > devs will be running in full over and over as they work. Assume > automated batch runs, which can be distributed over as many databases > and clients as you like. > > For small test subsets that are being used during testing the equation > still doesn't change much: reset the wiki to known state, run the tests. > Keep it simple! > > -- brion > > > On Thursday, September 23, 2010, Dan Nessett wrote: >> On Wed, 22 Sep 2010 12:30:35 -0700, Brion Vibber wrote: >> >>> On Wed, Sep 22, 2010 at 11:09 AM, Dan Nessett >>> wrote: >>> >>>> Some have mentioned the possibility of using the wiki family logic to >>>> help achieve these objectives. Do you have any thoughts on this? If >>>> you think it is a good idea, how do we find out more about it? >>>> >>>> >>> I'd just treat it same as any other wiki. Whether you're running one >>> or multiple wiki instances out of one copy of the code base, it just >>> doesn't make a difference here. >>> >>> -- brion >> >> Not to hammer this point to hard, but there is another reason for >> supporting switching in of temporary resources. >> >> We have one worked example of a Selenium test - PagedTiffHandler. >> During the course of its execution, it uploads some tiff files to test >> some of its functionality. Currently, once an image is uploded, there >> is no way to completely delete it (short of drastic administrative >> action involving raw database manipulation). So, running >> PagedTiffHandler twice doesn't work unless you switch in temporary >> resources on each run that are then deleted at the end of the run. >> >> -- >> -- Dan Nessett >> >> >> ___ Wikitech-l mailing list >> Wikitech-l@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l >> I am very much in favor of keeping it simple. I think the issue is whether we will support more than one regression test (or individual test associated with a regression test) running concurrently on the same test wiki. If not, then I agree, no switching logic is necessary. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Wed, 22 Sep 2010 12:30:35 -0700, Brion Vibber wrote: > On Wed, Sep 22, 2010 at 11:09 AM, Dan Nessett > wrote: > >> Some have mentioned the possibility of using the wiki family logic to >> help achieve these objectives. Do you have any thoughts on this? If you >> think it is a good idea, how do we find out more about it? >> >> > I'd just treat it same as any other wiki. Whether you're running one or > multiple wiki instances out of one copy of the code base, it just > doesn't make a difference here. > > -- brion Not to hammer this point to hard, but there is another reason for supporting switching in of temporary resources. We have one worked example of a Selenium test - PagedTiffHandler. During the course of its execution, it uploads some tiff files to test some of its functionality. Currently, once an image is uploded, there is no way to completely delete it (short of drastic administrative action involving raw database manipulation). So, running PagedTiffHandler twice doesn't work unless you switch in temporary resources on each run that are then deleted at the end of the run. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Wed, 22 Sep 2010 12:30:35 -0700, Brion Vibber wrote: > On Wed, Sep 22, 2010 at 11:09 AM, Dan Nessett > wrote: > >> Some have mentioned the possibility of using the wiki family logic to >> help achieve these objectives. Do you have any thoughts on this? If you >> think it is a good idea, how do we find out more about it? >> >> > I'd just treat it same as any other wiki. Whether you're running one or > multiple wiki instances out of one copy of the code base, it just > doesn't make a difference here. > > -- brion Oh. Well, perhaps we don't agree exactly 100%. As you suggest, let's step back a bit. Once we get the selenium framework working I assume it will be used for a regression test. This will comprise a set of individual tests. Generally, these tests will write into the wiki db (some may not, but many will). To ensure test reproducibility, the state of the wiki should be the same each time one of these individual tests runs. But, there is a problem. With parserTests, each individual test runs serially. That is fine for parserTests, since (I just ran this on my machine) while there are 610 individual tests, each takes about .08 seconds to run (on average). So, on my machine the whole parserTest takes about 48 seconds. Selenium tests are far more heavy-weight. A rough ball-park figure is each takes about 10 seconds to run (this does not include the time it would take to setup and tear down a "clean wiki"). So, a selenium-based regression test comprising 180 individual tests would take around 30 minutes. Not too bad. But, things are a bit more complicated. Each individual test runs multiple times, once for every browser/OS combination chosen for the regression test. For example, right now there are 13 configured browser/ OS combinations on the WMF Selenium Grid (see http:// grid.tesla.usability.wikimedia.org/console). So even if you only test 4 of these browser/OS configurations, the regression test (if individual tests run serially) would take 2 hours. If you test 8 of them, it would take 4 hours. This is starting to get onerous. If an individual developer wishes to ensure his modifications don't break things before committing his changes, then waiting 4 hours for a regression test to complete is a pretty heavy penalty. Generally, very few will pay the price. So, running the individual tests of a selenium-based regression test serially is not very attractive. This means you need to achieve some concurrency in the regression test. Since individual tests may interfere with each other, you need a way to protect them from each other. This is what the switching functionality is for. You switch in a base set of temporary resources for each test (or perhaps more likely for a particular test suite comprised of mulitple individual tests) consisting of a db, images directory, etc. This allows tests to run without interfering with each other. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Wed, 22 Sep 2010 11:00:53 -0700, Brion Vibber wrote: > Hmm, I think you guys are overthinking the details on this; let's step > back a level. > > When you're running tests, you have these tasks: * create a blank-slate > wiki to run tests in * populate the empty wiki with known data * run > tests on this wiki in a known state * clean up in some way > > The parserTests system is designed with some particular constraints: * > run within the local MediaWiki test instance a developer already has, > without altering its contents > * run all tests in a single command-line program * expose no in-test > data to the outside world > > It can do this very easily using temporary tables and a temporary > directory because it only needs to work within that one single test > process; once it's done, it's done and all the temporary data can be > discarded. Nothing needs to be kept across processes or exposed to > clients. > > For Selenium tests, you have a very different set of constraints: * test > data must be exposed to web clients over a web server * test data must > be retained across multiple requests > > The simplest way to accomplish this is to have a dedicated, web-exposed > test wiki instance. A test run would go like this: * (re)initialize test > wiki into known state * run series of tests > > I'd recommend not trying to use the parserTest harness/initialization > for this; it'll be a *lot* simpler to just script creation of a fresh > wiki. (drop database, create database, slurp tables, run update.php) > > Some non-destructive tests can always be run on any existing instance -- > and probably should be! -- and some 'active' tests will be freely > runnable on existing instances that are used for development and > testing, but if you want to work with a blank slate wiki exposed to web > clients, keep things simple and just make a dedicated instance. > > -- brion > > > On Wed, Sep 22, 2010 at 9:47 AM, Dan Nessett wrote: > >> On Wed, 22 Sep 2010 15:49:40 +0200, Markus Glaser wrote: >> >> > Hi, >> > >> > here are my thoughts about phpunit and selenium testing. >> > >> >> The wiki under test is set up with a master database consisting of a >> >> single objectcache table. The entries of this table specify a test >> >> run identifier as primary key and temporary resource identifiers as >> >> dependent fields. >> > If I understand this correctly, this would not allow to test any >> > wikis that are running on live sites, e.g. intranet wikis. While I >> > agree that regression testing on live sites is not a good idea, I >> > kind of like the notion that after setting up a wiki with all the >> > extensions I like to have, I could do some sort of "everything up and >> > running"-test. With the concept of using separate testing databases >> > and resources, this would be possible without interference with the >> > actual data and could even be done at intervals during, say, >> > maintenance periods. >> >> The problem with testing live sites is tests may alter wiki data >> (consequently, test run reproducibility becomes a problem). If all of >> the tests are read-only, then that isn't a problem, but it means >> developing a whole set of tests that conform to that constraint. >> >> Nevertheless, it wouldn't be hard to design the switching mechanism to >> allow the testing of live sites. There could be an option in test setup >> and cleanup that effectively says "don't switch-in/clean-up temporary >> resources." Personally, I think use of this option is dangerous, but it >> wouldn't be hard to implement. >> >> >> Setup of a test run requires the creation of the test run temporary >> >> resources and a entry in the objectcache table. >> > Are there already mechanisms for this? I haven't done too much work >> > with the objectcache. This is where memcached data is stored, right? >> > So how do I get the data that is needed? This question leads me to >> > another one: How do I get the testind database and resources? As I >> > see this, it should be part of the testing framework to be able to >> > produce the set of data needed from a "normal" MW installation. The >> > whole mechanism would actually be something like a backup, so we >> > might look into any existing solutions for that. >> >> We should use the existing ObjectCache class to manage the object >> cache. However, if there exists some switching-in code, I doubt it has >> cor
Re: [Wikitech-l] using parserTests code for selenium test framework
> I just looked at SqlBagOStuff. It already has entry expiration logic. > So, it seems perfect for use by the switch-in/clean-up functionality. I spoke too soon. Expired entries are deleted automatically when any entry is referenced. Unfortunately, that means there is no opportunity to reclaim the temporary resources identified in the entry before its state is lost. This is a problem. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Wed, 22 Sep 2010 18:57:12 +0200, Bryan Tong Minh wrote: > On Wed, Sep 22, 2010 at 6:47 PM, Dan Nessett wrote: >> >> I think the object cache and memcached are alternative ways of storing >> persistent data. (I also am not an expert in this, so I could be >> wrong). My understanding is memcached uses the memcached daemon >> (http:// memcached.org/), while the object cache uses the underlying >> database. If so, then memcached data disappears after a system crash or >> power outage, whereas object cache data should survive. >> >> > There is no ObjectCache class. Basically, there is a common base class > called BagOStuff and from that various classes for various backends are > defined such as SqlBagOStuff and MemcacheBagOStuff. To the code outside > that class, there is no visible difference. > > > Bryan I just looked at SqlBagOStuff. It already has entry expiration logic. So, it seems perfect for use by the switch-in/clean-up functionality. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Wed, 22 Sep 2010 19:35:31 +0200, Roan Kattouw wrote: > 2010/9/22 Dan Nessett : >> How does memcached fit into this? When I looked at BagOStuff, I didn't >> find a MemcacheBagOStuff class. Is it defined elsewhere? >> > Either memcached.php, MemCached.php or MWMemcached.php, I forget. The > class name is MWMemcached. > > Roan Kattouw (Catrope) Found it. It's in memcached-client.php. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Wed, 22 Sep 2010 18:57:12 +0200, Bryan Tong Minh wrote: > On Wed, Sep 22, 2010 at 6:47 PM, Dan Nessett wrote: >> >> I think the object cache and memcached are alternative ways of storing >> persistent data. (I also am not an expert in this, so I could be >> wrong). My understanding is memcached uses the memcached daemon >> (http:// memcached.org/), while the object cache uses the underlying >> database. If so, then memcached data disappears after a system crash or >> power outage, whereas object cache data should survive. >> >> > There is no ObjectCache class. Basically, there is a common base class > called BagOStuff and from that various classes for various backends are > defined such as SqlBagOStuff and MemcacheBagOStuff. To the code outside > that class, there is no visible difference. > > > Bryan Thanks for the clarification. I just looked at ObjectCache.php and it appears to provide a set of functions for accessing cache data of any type. Is this correct? How does memcached fit into this? When I looked at BagOStuff, I didn't find a MemcacheBagOStuff class. Is it defined elsewhere? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
that are copied during the switch-in will be configured as much as possible. For example, I think creating the base for PagedTiffHandler should start with a freshly installed wiki and upload multipage.tiff. The resulting db could be dumped and the images directory as well as other resources copied. The result could them be tar'd and uploaded as the base for the PagedTiffHandler test suite. This would relieve the test set up code of uploading multipage.tiff on each test run. We may even consider pre-installing the base db so that the switch-in code can use db functions to clone it, rather than creating it from the base dump for each test run that uses it. The latter could take a significant amount of time, thereby severely slowing testing. >> After the test run completes, the testing application cleans up the >> test run by requesting the deletion of the temporary resources and the >> objectcache table entry associated with the test run. > In some cases, tests will not change any data, e.g. testing dynamic skin > elements in vector skin. Would it make sense not to tear down the > testing environment in that case in order to save some time when testing > repeatedly? I think, there is a conflict between performance and amount > of data, but who wins? There may be ways to make things more efficient by not immediately deleting the temporary resources. However, eventually we have to delete them. So, there is a question of where the temporary resources identifier is stored so it can be used later for a clean-up request. I was assuming the switch-in request occurs in test suite start-up and the clean-up request occurs in the test suite finishing code. But, we should consider alternative implementation strategies. > In general, it seems to me that we have some similarity with what is > called wiki family on mediawiki.org. One could see multiple testing > environments as a set of multiple wikis that share a common codebase > [1]. Does anybody have experience with wiki families and the object > cache rsp. memcached? I agree. (In fact, I mentioned this previously). No need to reinvent the wheel. > I am not sure whether we can use the same codebase as parsertests. I'd > rather think, parser tests are a special case of what we are sketching > here. On the other hand, I don't think it is a good idea to have two > separate approaches for very similar tasks in the code. Do you think it > would be feasible to separate the preparation part from both parser > tests and selenium tests and build both of them on a common ground? > The parserTests code creates some temporary tables in an existing wiki database. Currently, these tables are empty and prefixed with a static identifier. The requirements for parserTests and selenium tests are significantly different. While we may be able to learn from the parserTests code, we would have to change the parserTest code significantly in order to use it. I think it would actually be less work to start from scratch. Of course, as you mention, there may be code used to support wiki families that we could use without much modification. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Mon, 20 Sep 2010 22:32:24 +0200, Platonides wrote: > Dan Nessett wrote: >> On Sun, 19 Sep 2010 23:42:08 +0200, Platonides wrote: >>> You load originaldb.objectcache, retrieve the specific configuration, >>> and switch into it. >>> For supporting many sumyltaneous configurations, the keyname could >>> have the instance (whatever that cookie is set to) appended, although >>> those dynamic configurations make me a bit nervous. >> >> Well, this may work, but consider the following. >> >> A nightly build environment (and even a local developer test >> environment) tests the latest revision using a suite of regression >> tests. These tests exercise the same wiki code, each parametrized by: >> >> + Browser type (e.g., Firefox, IE, Safari, Opera) + Database (e.g., >> MySQL, Postgres, SQLite) + OS platform (e.g., Linux, BSD unix variant, >> Windows variant) >> >> A particular test environment may not support all permutations of these >> parameters (in particular a local developer environment may support >> only one OS), but the code mechanism for supporting the regression >> tests should. To ensure timely completion of these tests, they will >> almost certainly run concurrently. >> >> So, when a regression test runs, it must not only retrieve the >> configuration data associated with it, it must create a test run >> environment (e.g., a test db, a test images directory, test cache >> data). The creation of this test run environment requires an identifier >> somewhere so its resources may be reclaimed when the test run completes >> or after an abnormal end of the test run. >> >> Thus, the "originaldb" must not only hold configuration data with db >> keys identifying the particular test and its parameters, but also an >> identifier for the test run that can be used to reclaim resources if >> the test ends abnormally. The question is whether using a full wiki db >> for this purpose is advantageous or whether stripping out all of the >> other tables except the objectcache table is the best implementation >> strategy. > > Such originaldb would be empty for an instance used just for regression > testing and could in fact only contain the objectcache table. If it's a > developer machine he would use the originaldb for local testing, but a > nigthly would not need to (in fact, errors trying to access those > missing tables would be useful for detecting errors in the isolating > system). Sounds reasonable. Using this approach, here is how I see the logical flow of a test run. (NB: by a test run, I mean an execution of a test in the regression test set). The wiki under test is set up with a master database consisting of a single objectcache table. The entries of this table specify a test run identifier as primary key and temporary resource identifiers as dependent fields. Setup of a test run requires the creation of the test run temporary resources and a entry in the objectcache table. A cookie or other state is returned to the testing application. This state is provided by the testing application during the test run on each request to the wiki under test. When a request is sent to the wiki under test, very early in the request processing (e.g., immediately after LocalSettings is processed) a hook is called with the provided state information as an argument that accesses the objectcache table. The extension function handling the hook switches in the temporary resources and returns. After the test run completes, the testing application cleans up the test run by requesting the deletion of the temporary resources and the objectcache table entry associated with the test run. In order to handle prematurely abandoned test runs, the objectcache table entry probably needs an entry that specifies its lifetime. If this lifetime expires, the temporary resources associated with the entry are reclaimed and the entry is delete. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Sun, 19 Sep 2010 23:42:08 +0200, Platonides wrote: > Dan Nessett wrote: >> Platonides wrote: >>> Dan Nessett wrote: >>>>> What about memcached? >>>>> (that would be a key based on the original db name) >>>> >>>> The storage has to be persistent to accommodate wiki crashes (e.g., >>>> httpd crash, server OS crash, power outage). It might be possible to >>>> use memcachedb, but as far as I am aware that requires installing >>>> Berkeley DB, which complicated deployment. >>>> >>>> Why not employ the already installed DB software used by the wiki? >>>> That provides persistent storage and requires no additional software. >>> >>> My original idea was to use whatever ObjectCache the wiki used, but it >>> could be forced to use the db as backend (that's the objectcache >>> table). >> >> My familiarity with the ObjectCache is casual. I presume it holds data >> that is set on particular wiki access requests and that data is then >> used on subsequent requests to make them more efficient. If so, then >> using a common ObjectCache for all concurrent test runs would cause >> interference between them. To ensure such interference doesn't exist, >> we would need to switch in a per-test-run ObjectCache (which takes us >> back to the idea of using a per-test-run db, since the ObjectCache is >> implemented using the objectcache table). > > You load originaldb.objectcache, retrieve the specific configuration, > and switch into it. > For supporting many sumyltaneous configurations, the keyname could have > the instance (whatever that cookie is set to) appended, although those > dynamic configurations make me a bit nervous. Well, this may work, but consider the following. A nightly build environment (and even a local developer test environment) tests the latest revision using a suite of regression tests. These tests exercise the same wiki code, each parametrized by: + Browser type (e.g., Firefox, IE, Safari, Opera) + Database (e.g., MySQL, Postgres, SQLite) + OS platform (e.g., Linux, BSD unix variant, Windows variant) A particular test environment may not support all permutations of these parameters (in particular a local developer environment may support only one OS), but the code mechanism for supporting the regression tests should. To ensure timely completion of these tests, they will almost certainly run concurrently. So, when a regression test runs, it must not only retrieve the configuration data associated with it, it must create a test run environment (e.g., a test db, a test images directory, test cache data). The creation of this test run environment requires an identifier somewhere so its resources may be reclaimed when the test run completes or after an abnormal end of the test run. Thus, the "originaldb" must not only hold configuration data with db keys identifying the particular test and its parameters, but also an identifier for the test run that can be used to reclaim resources if the test ends abnormally. The question is whether using a full wiki db for this purpose is advantageous or whether stripping out all of the other tables except the objectcache table is the best implementation strategy. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Sun, 19 Sep 2010 02:47:00 +0200, Platonides wrote: > Dan Nessett wrote: >>> What about memcached? >>> (that would be a key based on the original db name) >> >> The storage has to be persistent to accommodate wiki crashes (e.g., >> httpd crash, server OS crash, power outage). It might be possible to >> use memcachedb, but as far as I am aware that requires installing >> Berkeley DB, which complicated deployment. >> >> Why not employ the already installed DB software used by the wiki? That >> provides persistent storage and requires no additional software. > > My original idea was to use whatever ObjectCache the wiki used, but it > could be forced to use the db as backend (that's the objectcache table). My familiarity with the ObjectCache is casual. I presume it holds data that is set on particular wiki access requests and that data is then used on subsequent requests to make them more efficient. If so, then using a common ObjectCache for all concurrent test runs would cause interference between them. To ensure such interference doesn't exist, we would need to switch in a per-test-run ObjectCache (which takes us back to the idea of using a per-test-run db, since the ObjectCache is implemented using the objectcache table). -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Fri, 17 Sep 2010 19:13:33 +, Dan Nessett wrote: > On Fri, 17 Sep 2010 18:40:53 +0000, Dan Nessett wrote: > >> I have been tasked to evaluate whether we can use the parserTests db >> code for the selenium framework. I just looked it over and have serious >> reservations. I would appreciate any comments on the following >> analysis. >> >> The environment for selenium tests is different than that for >> parserTests. It is envisioned that multiple concurrent tests could run >> using the same MW code base. Consequently, each test run must: >> >> + Use a db that if written to will not destroy other test wiki >> information. >> + Switch in a new images and math directory so any writes do not >> interfere with other tests. >> + Maintain the integrity of the cache. >> >> Note that tests would *never* run on a production wiki (it may be >> possible to do so if they do no writes, but safety considerations >> suggest they should always run on a test data, not production data). In >> fact production wikis should always retain the setting >> $wgEnableSelenium = false, to ensure selenium test are disabled. >> >> Given this background, consider the following (and feel free to comment >> on it): >> >> parserTests temporary table code: >> >> A fixed set of tables are specified in the code. parserTests creates >> temporary tables with the same name, but using a different static >> prefix. These tables are used for the parserTests run. >> >> Problems using this approach for selenium tests: >> >> + Selenium tests on extensions may require use of extension specific >> tables, the names of which cannot be elaborated in the code. >> >> + Concurrent test runs of parserTests are not supported, since the >> temporary tables have fixed names and therefore concurrent writes to >> them by parallel test runs would cause interference. >> >> + Clean up from aborted runs requires dropping fossil tables. But, if a >> previous run tested an extension with extension-specific tables, there >> is no way for a test of some other functionality to figure out which >> tables to drop. >> >> For these reasons, I don't think we can reuse the parserTests code. >> However, I am open to arguments to the contrary. > > After reflection, here are some other problems. > > + Some tests assume the existence of data in the db. For example, the > PagedTiffHandler tests assume the image Multipage.tiff is already > loaded. However, this requires an entry in the image table. You could > modify the test to clone the existing image table, but that means you > have problems with: > > + Some tests assume certain data is *not* in the db. PagedTiffHandler > has tests that upload images. These cannot already be in the images > table. So, you can't simply clone the images table. > > All of this suggests to me that a better strategy is: > > + When the test run begins, clone a db associated with the test suite. > > + Switch the wiki to use this db and return a cookie or some other state > information that identifies this test run configuration. > > + When the test suite runs, each wiki access supplies this state so the > wiki code can switch in the correct db. > > + Cleanup of test runs requires removing the cloned db. > > + To handled aborted runs, there needs to be a mechanism to time out > cloned dbs and the state associated with the test run. Regardless of how we implement the persistent storage for managing test runs, there needs to be a way to trigger it use. To minimize the changes to core, we need a hook that runs after processing LocalSettings (and by implication DefaultSettings), but before any wiki state is accessed (e.g., before accessing the db, the images directory, any cached data). I looked at the existing hooks, but so far have not found one that appears suitable. So, either we need to identify an appropriate existing hook, or we need to add a hook that meets the requirements. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Sun, 19 Sep 2010 00:28:42 +0200, Platonides wrote: >> Where to store the data is an open question, one that requires >> consultation with others. However, here are some thoughts: >> >> + The data must be persistent. If the wiki crashes for some reason, >> there may be cloned dbs and test-specific copies of images and >> images/math hanging around. (Depending how we handle the cache >> information, there may also be fossil cache data). This requires >> cleanup after a wiki crash. >> >> + It would be possible to store the data in a file or in a master db >> table. Which is best (or if something else is better) is a subject for >> discussion. > > What about memcached? > (that would be a key based on the original db name) The storage has to be persistent to accommodate wiki crashes (e.g., httpd crash, server OS crash, power outage). It might be possible to use memcachedb, but as far as I am aware that requires installing Berkeley DB, which complicated deployment. Why not employ the already installed DB software used by the wiki? That provides persistent storage and requires no additional software. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Keeping record of imported licensed text
On Fri, 10 Sep 2010 23:11:27 +, Dan Nessett wrote: > We are currently attempting to refactor some specific modifications to > the standard MW code we use (1.13.2) into an extension so we can upgrade > to a more recent maintained version. One modification we have keeps a > flag in the revisions table specifying that article text was imported > from WP. This flag generates an attribution statement at the bottom of > the article that acknowledges the import. > > I don't want to start a discussion about the various legal issues > surrounding text licensing. However, assuming we must acknowledge use of > licensed text, a legitimate technical issue is how to associate state > with an article in a way that records the import of licensed text. I > bring this up here because I assume we are not the only site that faces > this issue. > > Some of our users want to encode the attribution information in a > template. The problem with this approach is anyone can come along and > remove it. That would mean the organization legally responsible for the > site would entrust the integrity of site content to any arbitrary > author. We may go this route, but for the sake of this discussion I > assume such a strategy is not viable. So, the remainder of this post > assumes we need to keep such licensing state in the db. > > After asking around, one suggestion was to keep the licensing state in > the page_props table. This seems very reasonable and I would be > interested in comments by this community on the idea. Of course, there > has to be a way to get this state set, but it seems likely that could be > achieved using an extension triggered when an article is edited. > > Since this post is already getting long, let me close by asking whether > support for associating licensing information with articles might be > useful to a large number of sites. If so, the perhaps it belongs in the > core. One thing I haven't seen so far (probably because it doesn't belong on Wikitech) is a discussion of the policy requirements. In open source software development, you have to carry forward licenses even if you substantially change the code content. The only way around this is a "clean room" implementation (e.g., how BSD Unix got around AT&T's original licensing for Unix). Is this also true for textual content? If so, then once you import such content into an article you are obliged to carry forward any licensing conditions on that import on for all subsequent revisions. Where is the proper place to discuss these kinds of questions? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Sat, 18 Sep 2010 00:53:04 +0200, Platonides wrote: > >> + Switch the wiki to use this db and return a cookie or some other >> state information that identifies this test run configuration. > > I think you mean for remote petitions, not just for internal queries, > where do you expect to store that data? Not sure what you mean by remote petitions. Selenium requests always come through the web portal from the selenium server. So, no internal queries are involved. Where to store the data is an open question, one that requires consultation with others. However, here are some thoughts: + The data must be persistent. If the wiki crashes for some reason, there may be cloned dbs and test-specific copies of images and images/math hanging around. (Depending how we handle the cache information, there may also be fossil cache data). This requires cleanup after a wiki crash. + It would be possible to store the data in a file or in a master db table. Which is best (or if something else is better) is a subject for discussion. We may be able to use the mechanisms in the code that supports access to different language versions of a wiki (e.g., Wikipedia) using the same code. I am not familiar with these mechanisms, so this approach requires help from someone who is. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Fri, 17 Sep 2010 21:05:12 +0200, Platonides wrote: > Dan Nessett wrote: >> Given this background, consider the following (and feel free to comment >> on it): >> >> parserTests temporary table code: >> >> A fixed set of tables are specified in the code. parserTests creates >> temporary tables with the same name, but using a different static >> prefix. These tables are used for the parserTests run. >> >> Problems using this approach for selenium tests: >> >> + Selenium tests on extensions may require use of extension specific >> tables, the names of which cannot be elaborated in the code. > > The extensions could list their table names. No problem there. > > >> + Concurrent test runs of parserTests are not supported, since the >> temporary tables have fixed names and therefore concurrent writes to >> them by parallel test runs would cause interference. > > So it gets changed to a random name with a fixed prefix... What concerns > me is > >> + Clean up from aborted runs requires dropping fossil tables. But, if a >> previous run tested an extension with extension-specific tables, there >> is no way for a test of some other functionality to figure out which >> tables to drop. > > Run a script dropping all tables with a fixed prefix (the shared part of > the tests) when you have no tests running. > >> For these reasons, I don't think we can reuse the parserTests code. >> However, I am open to arguments to the contrary. > > There may be other issues with that code, and using a separate db would > be preferable if you have enough permissions, but this doesn't seem like > real problems. > > What concerns me is that Oracle is using (r58669) a different prefix for > the parsertests table. If it has some restriction on [medium-large] > table names, there may not be possible to run the tests there using the > long table names that we could produce. The strategy you suggest is reasonable. But, I think it requires significant changes to the parserTests code. The question then is: is it simpler to modify this code or just write something new? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Fri, 17 Sep 2010 18:40:53 +, Dan Nessett wrote: > I have been tasked to evaluate whether we can use the parserTests db > code for the selenium framework. I just looked it over and have serious > reservations. I would appreciate any comments on the following analysis. > > The environment for selenium tests is different than that for > parserTests. It is envisioned that multiple concurrent tests could run > using the same MW code base. Consequently, each test run must: > > + Use a db that if written to will not destroy other test wiki > information. > + Switch in a new images and math directory so any writes do not > interfere with other tests. > + Maintain the integrity of the cache. > > Note that tests would *never* run on a production wiki (it may be > possible to do so if they do no writes, but safety considerations > suggest they should always run on a test data, not production data). In > fact production wikis should always retain the setting $wgEnableSelenium > = false, to ensure selenium test are disabled. > > Given this background, consider the following (and feel free to comment > on it): > > parserTests temporary table code: > > A fixed set of tables are specified in the code. parserTests creates > temporary tables with the same name, but using a different static > prefix. These tables are used for the parserTests run. > > Problems using this approach for selenium tests: > > + Selenium tests on extensions may require use of extension specific > tables, the names of which cannot be elaborated in the code. > > + Concurrent test runs of parserTests are not supported, since the > temporary tables have fixed names and therefore concurrent writes to > them by parallel test runs would cause interference. > > + Clean up from aborted runs requires dropping fossil tables. But, if a > previous run tested an extension with extension-specific tables, there > is no way for a test of some other functionality to figure out which > tables to drop. > > For these reasons, I don't think we can reuse the parserTests code. > However, I am open to arguments to the contrary. After reflection, here are some other problems. + Some tests assume the existence of data in the db. For example, the PagedTiffHandler tests assume the image Multipage.tiff is already loaded. However, this requires an entry in the image table. You could modify the test to clone the existing image table, but that means you have problems with: + Some tests assume certain data is *not* in the db. PagedTiffHandler has tests that upload images. These cannot already be in the images table. So, you can't simply clone the images table. All of this suggests to me that a better strategy is: + When the test run begins, clone a db associated with the test suite. + Switch the wiki to use this db and return a cookie or some other state information that identifies this test run configuration. + When the test suite runs, each wiki access supplies this state so the wiki code can switch in the correct db. + Cleanup of test runs requires removing the cloned db. + To handled aborted runs, there needs to be a mechanism to time out cloned dbs and the state associated with the test run. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] using parserTests code for selenium test framework
I have been tasked to evaluate whether we can use the parserTests db code for the selenium framework. I just looked it over and have serious reservations. I would appreciate any comments on the following analysis. The environment for selenium tests is different than that for parserTests. It is envisioned that multiple concurrent tests could run using the same MW code base. Consequently, each test run must: + Use a db that if written to will not destroy other test wiki information. + Switch in a new images and math directory so any writes do not interfere with other tests. + Maintain the integrity of the cache. Note that tests would *never* run on a production wiki (it may be possible to do so if they do no writes, but safety considerations suggest they should always run on a test data, not production data). In fact production wikis should always retain the setting $wgEnableSelenium = false, to ensure selenium test are disabled. Given this background, consider the following (and feel free to comment on it): parserTests temporary table code: A fixed set of tables are specified in the code. parserTests creates temporary tables with the same name, but using a different static prefix. These tables are used for the parserTests run. Problems using this approach for selenium tests: + Selenium tests on extensions may require use of extension specific tables, the names of which cannot be elaborated in the code. + Concurrent test runs of parserTests are not supported, since the temporary tables have fixed names and therefore concurrent writes to them by parallel test runs would cause interference. + Clean up from aborted runs requires dropping fossil tables. But, if a previous run tested an extension with extension-specific tables, there is no way for a test of some other functionality to figure out which tables to drop. For these reasons, I don't think we can reuse the parserTests code. However, I am open to arguments to the contrary. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Keeping record of imported licensed text
On Fri, 10 Sep 2010 23:11:27 +, Dan Nessett wrote: > We are currently attempting to refactor some specific modifications to > the standard MW code we use (1.13.2) into an extension so we can upgrade > to a more recent maintained version. One modification we have keeps a > flag in the revisions table specifying that article text was imported > from WP. This flag generates an attribution statement at the bottom of > the article that acknowledges the import. > > I don't want to start a discussion about the various legal issues > surrounding text licensing. However, assuming we must acknowledge use of > licensed text, a legitimate technical issue is how to associate state > with an article in a way that records the import of licensed text. I > bring this up here because I assume we are not the only site that faces > this issue. > > Some of our users want to encode the attribution information in a > template. The problem with this approach is anyone can come along and > remove it. That would mean the organization legally responsible for the > site would entrust the integrity of site content to any arbitrary > author. We may go this route, but for the sake of this discussion I > assume such a strategy is not viable. So, the remainder of this post > assumes we need to keep such licensing state in the db. > > After asking around, one suggestion was to keep the licensing state in > the page_props table. This seems very reasonable and I would be > interested in comments by this community on the idea. Of course, there > has to be a way to get this state set, but it seems likely that could be > achieved using an extension triggered when an article is edited. > > Since this post is already getting long, let me close by asking whether > support for associating licensing information with articles might be > useful to a large number of sites. If so, the perhaps it belongs in the > core. The discussion about whether to support license data in the database has settled down. There seems to be some support. So, I think the next step is to determine the best technical approach. Below I provide a strawman proposal. Note that this is only to foster discussion on technical requirements and approaches. I have nothing invested in the strawman. Implementation location: In an extension Permissions: include two new permissions - 1) addlicensedata, and 2) modifylicensedata. These are pretty self-explanatory. Sites that wish to give all users the ability to provide and modify licensing data would assign these permissions to everyone. Sites that wish to allow all users to add licensing data, but restrict those who are allowed to modify it, would give the first permission to everyone and the second to a limited group. Database schema: Add a "licensing" table to the db with the following columns - 1) revision_or_image, 2) revision_id, 3) image_id, 4) content_source, 5) license_id, 6) user_id. The first three columns identify the revision or image to which the licensing data is associated. I am not particularly adept with SQL, so there may be a better way to do this. The content_source column is a string that is a URL or other reference that specifies the source of the content under license. The license_id identifies the specific license for the content. The user_id identifies the user that added the licensing information. The user_id may be useful if a site wishes to allow someone who added the licensing information to delete or modify it. However, there are complications with this. Since IP addresses are easily spoofed, it would mean this entry should only be valid for logged in users. Add a "license" table with the following columns - 1) license_id, 2) license_text, 3) license name and 4) license_version. The license_id in the licensing table references rows in this table. One complication is when a page or image is reverted, the licensing table must be modified to reflect the current state. Data manipulation: The extension would use suitable hooks to insert, modify and render licensing data. Insertion and modification would probably use a relevant Edit Page or Article Management hook. Rendering would probably use a Page Rendering Hook. Page rendering: You probably don't want to dump licensing data directly onto a page. Instead, it is preferable to output a short licensing statement like: "Content on this page uses licensed content. For details, see licensing data." The phrase "licensing data" would be a link to a special page that accesses the licensing table and displays the license data associated with the page. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Selenium Framework - standard directory for selenium tests
Are there any standards for where to put selenium tests? Right now the Simple Selenium test is in phase3/maintenance/tests/selenium and the PagedTiffHandler selenium tests are in PagedTiffHandler/selenium. This suggests a convention of putting extension selenium test files in a sub- directory of the top-level directory named 'selenium'. Is that an official convention? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Keeping record of imported licensed text
We are currently attempting to refactor some specific modifications to the standard MW code we use (1.13.2) into an extension so we can upgrade to a more recent maintained version. One modification we have keeps a flag in the revisions table specifying that article text was imported from WP. This flag generates an attribution statement at the bottom of the article that acknowledges the import. I don't want to start a discussion about the various legal issues surrounding text licensing. However, assuming we must acknowledge use of licensed text, a legitimate technical issue is how to associate state with an article in a way that records the import of licensed text. I bring this up here because I assume we are not the only site that faces this issue. Some of our users want to encode the attribution information in a template. The problem with this approach is anyone can come along and remove it. That would mean the organization legally responsible for the site would entrust the integrity of site content to any arbitrary author. We may go this route, but for the sake of this discussion I assume such a strategy is not viable. So, the remainder of this post assumes we need to keep such licensing state in the db. After asking around, one suggestion was to keep the licensing state in the page_props table. This seems very reasonable and I would be interested in comments by this community on the idea. Of course, there has to be a way to get this state set, but it seems likely that could be achieved using an extension triggered when an article is edited. Since this post is already getting long, let me close by asking whether support for associating licensing information with articles might be useful to a large number of sites. If so, the perhaps it belongs in the core. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium Framework - test run configuration data
On Tue, 07 Sep 2010 17:39:19 +0200, Markus Glaser wrote: > Hi all, > > I suggest we have one static class variable Selenium::$settings which is > set up as an array. This array would be filled in some init function > from whatever sources we decide on. Then, internally, the configuration > mechanisms would not change anymore, and we could use the init method to > fill the settings from globals (as is) or ini files (as Mark propses). > Those who use the framework, however, would not have to rewrite their > code. > > Regards, > Markus > > Markus Glaser > Social Web Technologien > Leiter Softwareentwicklung > Hallo Welt! - Medienwerkstatt GmbH > __ > > Untere Bachgasse 15 > 93047 Regensburg > > Tel. +49 (0) 941 - 56 95 94 92 > Fax. +49 (0) 941 - 50 27 58 13 > > > www.hallowelt.biz > gla...@hallowelt.biz > > Sitz: Regensburg > Handelsregister: HRB 10467 > E.USt.Nr.: DE 253050833 > Geschäftsführer: > Anja Ebersbach, Markus Glaser, > Dr. Richard Heigl, Radovan Kubani > > > -Ursprüngliche Nachricht----- > Von: wikitech-l-boun...@lists.wikimedia.org > [mailto:wikitech-l-boun...@lists.wikimedia.org] Im Auftrag von dan > nessett Gesendet: Dienstag, 7. September 2010 17:20 An: Wikimedia > developers; Mark A. Hershberger Betreff: Re: [Wikitech-l] Selenium > Framework - test run configuration data > > I would be happy to coordinate with you. Up to this point I have been > working with Markus Glaser and that has gone pretty well. But, I would > welcome more discussion on issues, architecture and implementation > strategy for the Framework as well as making sure anything I do fits in > with what others are doing. > > Regards, > > Dan > > --- On Tue, 9/7/10, Mark A. Hershberger wrote: > >> From: Mark A. Hershberger Subject: Re: Selenium >> Framework - test run configuration data To: "Wikimedia developers" >> Cc: "Dan Nessett" >> Date: Tuesday, September 7, 2010, 8:04 AM Dan Nessett >> >> writes: >> >> > Either approach works. But, by going back and forth, >> it makes development >> > of functionality for the Framework difficult. >> >> I should also point out that what I was planning to do was not hidden. >> I wrote about these changes in my weekly report the Monday before I >> committed them (http://bit.ly/cqAcqz) and pointed to the weekly report >> from my Ohloh, Twitter, Identi.ca and Facebook accounts. >> >> Granted, this is not the same as posting to the mailing list, and for >> that I apologize. >> >> Looking back in the archives on gmane, it looks like you are very >> interested in MW testing. Since this is a large part of my focus >> currently as well, perhaps we should coordinate our work? >> >> Mark. >> >> -- >> http://hexmode.com/ >> >> Embrace Ignorance. Just don't get too attached. >> >> > > > > ___ Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l This seems like a reasonable strategy to me. Dan -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium Framework - test run configuration data
I would be happy to coordinate with you. Up to this point I have been working with Markus Glaser and that has gone pretty well. But, I would welcome more discussion on issues, architecture and implementation strategy for the Framework as well as making sure anything I do fits in with what others are doing. Regards, Dan --- On Tue, 9/7/10, Mark A. Hershberger wrote: > From: Mark A. Hershberger > Subject: Re: Selenium Framework - test run configuration data > To: "Wikimedia developers" > Cc: "Dan Nessett" > Date: Tuesday, September 7, 2010, 8:04 AM > Dan Nessett > writes: > > > Either approach works. But, by going back and forth, > it makes development > > of functionality for the Framework difficult. > > I should also point out that what I was planning to do was > not hidden. > I wrote about these changes in my weekly report the Monday > before I > committed them (http://bit.ly/cqAcqz) and pointed to the > weekly report > from my Ohloh, Twitter, Identi.ca and Facebook accounts. > > Granted, this is not the same as posting to the mailing > list, and for > that I apologize. > > Looking back in the archives on gmane, it looks like you > are very > interested in MW testing. Since this is a large part > of my focus > currently as well, perhaps we should coordinate our work? > > Mark. > > -- > http://hexmode.com/ > > Embrace Ignorance. Just don't get too attached. > ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium Framework - test run configuration data
On Mon, 06 Sep 2010 23:15:06 -0400, Mark A. Hershberger wrote: > Dan Nessett writes: > >> Last Friday, mah ripped out the globals and put the configuration >> information into the execute method of RunSeleniumTests.php with the >> comment "@todo Add an alternative where settings are read from an INI >> file." So, it seems we have dueling developers with contrary ideas >> about what is the best way to configure selenium framework tests. > > I'm opposed to increasing global variables and I think I understand > Tim's concern about configuring via a PHP file. > > I plan to start work on reading the configuration from an INI file > (*not* a PHP file). > >> Either approach works. But, by going back and forth, it makes >> development of functionality for the Framework difficult. > > I agree. > > The idea I was pursuing is to encapsulate configuration in a Selenium > object that (right now) RunSeleniumTests.php will set up. > > Platonides suggestion of a hook to provide configuration is also doable. > > Mark. I am pretty much agreeable to any solution that remains stable. One thing that may not be obvious is there may be configuration data over and above that specified on the RunSeleniumTests command line. For example, it is inconvenient to have to start up the selenium server before running RunSeleniumTests. (In the past I have frequently executed RunSeleniumTests only to get an error because I forgot to start the server). I supplied a patch to Markus recently that adds two options to the command line, one to start the server and the other to stop it (the patch supplies functionality only for *nix systems, which is probably why Markus has not committed it - there needs to be similar support for Windows). This functionality requires a directory path to the selenium server jar file. This is not something that a tester would normally supply on a command line. It is a system, not a test run parameter. So, I hope the INI file processing you are working on will allow the specification of such data. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Selenium Framework - test run configuration data
Back in June the Selenium Framework had a local configuration file called LocalSeleniumSettings.php. This was eliminated by Tim Starling in a 6/24 commit with the comment that it was an insecure concept. In that commit, new globals were added that controlled test runs. Last Friday, mah ripped out the globals and put the configuration information into the execute method of RunSeleniumTests.php with the comment "@todo Add an alternative where settings are read from an INI file." So, it seems we have dueling developers with contrary ideas about what is the best way to configure selenium framework tests. Should configuration data be exposed as globals or hidden in a local configuration file? Either approach works. But, by going back and forth, it makes development of functionality for the Framework difficult. I am working on code not yet submitted as a patch that now requires reworking because how to reference configuration data has changed. We need a decision that decides which of the two approaches to use. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Testing Framework
On Sat, 07 Aug 2010 23:30:16 -0400, Mark A. Hershberger wrote: > Dan Nessett writes: > >> I don't think walking through all the extensions looking for test >> subdirectories and then running all tests therein is a good idea. >> First, in a large installation with many extensions, this takes time >> and delays the test execution. > > Globbing for extensions/*/tests/TestSettings.php doesn't take long at > all. > > However I am looking at a way to test extensions independently of > installation. > > This means I can't depend on hooks or global variables, so I need > another way to find out if an extension has tests available. > >> Making the developer specify the extension or core tests to run on the >> RunSeleniumTests command line is irritating (at least, it would >> irritate me) > > No doubt. So why not allow per-user files to set this instead of using > LocalSettings.php? > > Mark. Testing uninstalled extensions may make sense for unit tests, but not for selenium tests. The latter exercise the code through a browser, so the extension must be installed for selenium testing. I'm not sure what are the advantages of "per-user" configuration files. For unit tests the tester directly accesses the code and so has direct access to LocalSettings. For selenium testing, we originally had a configuration file called SeleniumLocalSettings.php, but that was abandoned in favor of putting the configuration information in DefaultSettings and LocalSettings. As stated previously, selenium tests exercise MW code by accessing the wiki through a browser. I don't see how a 'per-user' configuration file would be integrated without introducing some serious security issues. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Testing Framework (was Selenium Framework - Question on coding conventions)
On Fri, 06 Aug 2010 12:38:39 -0400, Aryeh Gregor wrote: > On Thu, Aug 5, 2010 at 6:47 PM, Mark A. Hershberger > wrote: >> Can I suggest that the framework can see that an extension has tests >> simply by the presence of the /tests directory containing >> a *TestSuite.php file? > > IMO, the way parser tests do it is smarter. When you install the > extension, it adds the location of the test files to $wgParserTestFiles. > That way, only the tests associated with installed extensions will run. > If you want to force only particular tests to run all the time, you can > also modify the variable in LocalSettings.php as usual. We are doing something similar. In the extension require() file, the test suite is added to $wgAutoloadClasses. Right now the entry in $wgSeleniumTestSuites is pushed in LocalSettings. However, we could establish the convention that it is pushed in the extension require() file as well. Then all extensions with test suites would automatically load them. To tailor this, the entries in $wgSeleniumTestSuites could be removed in LocalSettings. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Testing Framework (was Selenium Framework - Question on coding conventions)
On Thu, 05 Aug 2010 18:47:58 -0400, Mark A. Hershberger wrote: > Markus Glaser writes: > >> 1) Where are the tests located? I suggest for core to put them into >> maintenance/tests/selenium. That is where they are now. For extensions >> I propse a similar structure, that is /tests/selenium. > > Sounds fine. > > In the same way, since maintenance/tests contains tests that should be > run using PHPUnit, we can say that /tests will contain > tests that should be run using PHPUnit. > >> Alternatively, we could use the word "Selenium" somewhere in there in >> order to be able to divide between unit and selenium tests. > > I think putting them in the selenium directory (or the “Se” directory) > is sufficient. > >> 3) How does the framework know there are tests? > > Can I suggest that the framework can see that an extension has tests > simply by the presence of the /tests directory containing > a *TestSuite.php file? > > The /tests/TestSuite.php file should define a > class using the name TestSuite which has a static method > suite(). See the PHPUnit documentation at http://bit.ly/b9L50r for how > this is set up. > > This static suite() method should take care of letting the autoloader > know about any test classes so the test classes are only available > during testing. > > So, for your example using PagedTiffHandler, there would be the files: > > PagedTiffHandler/tests/PagedTiffHandlerTestSuite.php > PagedTiffHandler/tests/PagedTiffHandlerUploadsTestSuite.php > >> 4) Which tests should be executed? > > By default all the test suites in /tests should be run. > > It is should be possible to specify which particular test to run by > using whatever command line arguments to the CLI. > > This seems better to me than defining a new global. If some tests > should only be run rarely, that information can be put in the TestSuite > class for te extension. > > In this way, I think it is possible to remove all the $wgSelenium* > variables from the DefaultSettings.php file. > > (I plan to do something similar with the $wgParserTest* variables as > well — these sorts of things don't seem like they belong in Core.) > > Mark. I don't think walking through all the extensions looking for test subdirectories and then running all tests therein is a good idea. First, in a large installation with many extensions, this takes time and delays the test execution. Second, if a developer is working on a particular extension or part of the core, it will be common to run the tests associated with that for regression purposes. Making the developer specify the extension or core tests to run on the RunSeleniumTests command line is irritating (at least, it would irritate me). Specifying the test suite(s) to be run in LocalSettings.php is a set and forget approach that allows the developer to get on with the work. However, I do agree that the number of global variables associated with the selenium framework is getting large and has the potential of growing over time. One solution is to use a multi-dimensional associative array (much like $wgGroupPermissions). We could use a global variable $wgSelenium and move all selenium framework values into it. For example: $wgSelenium['wiki']['host'] = 'localhost'; $wgSelenium['wiki']['wikiurl'] = false; $wgSelenium['wiki']['loginname'] = 'Wikiuser; $wgSelenium['wiki']['password'] = ''; $wgSelenium['server']['port'] = ; etc. The only global we may wish to keep separate is $wgEnableSelenium, since it specifies whether $wgSelenium is used. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework - Fix available in Revision 67575?
On Mon, 07 Jun 2010 17:37:49 -0700, Michelle Knight wrote: > Hi, > > > I am following up on a thread from May 27th where there is a bug in run > Selenium Tests. I was wondering if the fix for Run Selenium Tests is > available in Revision 67575 found at > http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/ PagedTiffHandler/tests/ > <http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/ PagedTiffHandler/tests/%20>. > In the meantime I will keep all the local configuration data in the > RunSeleniumTests as instructed.* *I was about to copy and paste the php > tests in Selenium IDE to see what would happen and if I could run from > there. > > Michelle Knight > > > > Message: 6 > Date: Thu, 27 May 2010 17:47:48 + (UTC) From: Dan Nessett > Subject: Re: [Wikitech-l] Selenium testing > framework- Firefox browsers >compatible > To: wikitech-l@lists.wikimedia.org > Message-ID: Content-Type: text/plain; > charset=UTF-8 > > On Wed, 26 May 2010 17:11:33 -0700, Michelle Knight wrote: > >> Hi Dan, >> >> There is a list of browsers compatible with Selenium (See >> http://seleniumhq.org/about/<http://seleniumhq.org/about/ platforms.html#browsers> > platforms.html#browsers<http://seleniumhq.org/about/ platforms.html#browsers>). > The page states >> that Selenium works with Firefox 2+ when a Linux OS is used (I think >> Ubuntu would fall under this category ). >> >> I am using Firefox 3.5.9 on Ubuntu 9.10 . I have been finishing >> another >> project (my grandfather visited me in Oregon from Ohio) and have not >> played with the at the Selenium Framework since May 14th. I will let >> you know if I see the error messages. >> >> Michelle Knight >> >> >> >> Message: 5 >> Date: Tue, 18 May 2010 17:44:03 + (UTC) From: Dan Nessett >> Subject: Re: [Wikitech-l] Selenium testing >> framework To: wikitech-l@lists.wikimedia.org Message-ID: >> Content-Type: text/plain; charset=UTF-8 >> >> On Tue, 18 May 2010 19:27:38 +0200, Markus Glaser wrote: >> >>> Hi Dan, >>> >>> I had these error messages once when I used Firefox 3.6 for testing. >>> Until recently, Selenium did not support this browser. Apparently now >>> they do, but I did not have a chance to test this yet. So the solution >>> for me was to point Selenium to a Firefox 3.5. >>> >>> Cheers, >>> Markus >> >> My OS is Ubuntu 8.04. The version of Firefox is 3.0.19. Since Ubuntu >> automatically updates versions of its software, I assume this is the >> most up-to-date. >> >> Is there a list of browser versions compatible with selenium? > > *Thanks for the pointer to the list, Michelle. As it turned out there > was bug in RunSeleniumTests that accessed global data before > LocalSeleniumSettings was included. Markus has fixed this problem and is > testing it before checking it in to the repository. Before this fix is > available, you should put all your local configuration data in > RunSeleniumTests. > > Regards, > > Dan* > > -- > -- Dan Nessett > Regards, > > Dan Hi Michelle, Yes, r67575 should have the fix that allows you to put configuration data in LocalSeleniumSettings. Also, it has a bug fix (integrated at r67572) that supports login for MW versions < 1.15.2. Regards, Dan Nessett -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] revision of Nuke that works with postgres
The Nuke extension doesn't work with postgres (https:// bugzilla.wikimedia.org/show_bug.cgi?id=23600). Is there a revision that contains a version that does? Right now (for 1.13.2) the snapshot returned by the Nuke extension page is: r37906. This produces the error given in the bug ticket. Regards, -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework- Firefox browsers compatible
On Wed, 26 May 2010 17:11:33 -0700, Michelle Knight wrote: > Hi Dan, > > There is a list of browsers compatible with Selenium (See > http://seleniumhq.org/about/platforms.html#browsers ). The page states > that Selenium works with Firefox 2+ when a Linux OS is used (I think > Ubuntu would fall under this category ). > > I am using Firefox 3.5.9 on Ubuntu 9.10 . I have been finishing another > project (my grandfather visited me in Oregon from Ohio) and have not > played with the at the Selenium Framework since May 14th. I will let you > know if I see the error messages. > > Michelle Knight > > > > Message: 5 > Date: Tue, 18 May 2010 17:44:03 + (UTC) From: Dan Nessett > Subject: Re: [Wikitech-l] Selenium testing > framework To: wikitech-l@lists.wikimedia.org > Message-ID: Content-Type: text/plain; > charset=UTF-8 > > On Tue, 18 May 2010 19:27:38 +0200, Markus Glaser wrote: > >> Hi Dan, >> >> I had these error messages once when I used Firefox 3.6 for testing. >> Until recently, Selenium did not support this browser. Apparently now >> they do, but I did not have a chance to test this yet. So the solution >> for me was to point Selenium to a Firefox 3.5. >> >> Cheers, >> Markus > > My OS is Ubuntu 8.04. The version of Firefox is 3.0.19. Since Ubuntu > automatically updates versions of its software, I assume this is the > most up-to-date. > > Is there a list of browser versions compatible with selenium? Thanks for the pointer to the list, Michelle. As it turned out there was bug in RunSeleniumTests that accessed global data before LocalSeleniumSettings was included. Markus has fixed this problem and is testing it before checking it in to the repository. Before this fix is available, you should put all your local configuration data in RunSeleniumTests. Regards, Dan -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Sun, 02 May 2010 02:20:06 +0200, Markus Glaser wrote: > Hi everybody, > > at the Wkimedia Developers' Workshop, I introduced a Selenium testing > framework for MediaWiki. Since it has now been promoted to > maintenance/tests, I have provided some initial information it on > http://www.mediawiki.org/wiki/SeleniumFramework . I would be very happy > about comments and ideas for further improvement. Also, if you intend to > use the framework for your tests, please let me know. I will be happy to > assist. > > Regards, > Markus Glaser > > __ > > Social Web Technologien > Leiter Softwareentwicklung > Hallo Welt! - Medienwerkstatt GmbH > > __ > > Untere Bachgasse 15 > 93047 Regensburg > > Tel. +49 (0) 941 - 56 95 94 - 92 > > www.hallowelt.biz<http://www.hallowelt.biz/> > gla...@hallowelt.biz<mailto:gla...@hallowelt.biz> > > Sitz: Regensburg > Handelsregister: HRB 10467 > E.USt.Nr.: DE 253050833 > Geschäftsführer: > Anja Ebersbach, Markus Glaser, > Dr. Richard Heigl, Radovan Kubani > > __ Hi Markus, Despite my initial problems getting the Selenium Framework to run, I think it is a great start. Now that I have the PagedTiffHandler working, here is some feedback on the current framework: + When I svn up ../tests (or any ancestor directory), the local changes I make to RunSeleniumTests cause a local conflict error. Eventually, many of the configuration changes I made should appear in LocalSeleniumSettings, but it isn't clear that is possible for all of them. For example, I change the commented out set_include_path to include my local PHP/PEAR directory. Can this be set in LocalSeleniumSettings? Another difference is the include_once() for each test suite. Is it possible to move these into LocalSeleniumSettings? + It appears there is no way to tell RunSeleniumTests to use a selenium server port other than . It would be useful to have a -port parameter on RunSeleniumServer for this. For example, if there are multiple developers working on the same machine, they probably need to use different selenium servers differentiated by different port numbers. I don't mind working on both of these issues, but since you are the original architect of the framework, it is probably best for you to comment on them first and perhaps suggest what you consider to be the best approach to their resolution. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Mon, 17 May 2010 20:16:35 +, Dan Nessett wrote: > On Mon, 17 May 2010 19:11:21 +0000, Dan Nessett wrote: > >> During the meeting last Friday, someone (I sorry, I don't remember who) >> mentioned he had created a test that runs with the currently checked in >> selenium code. Is that test code available somewhere (it doesn't appear >> to be in the current revision)? > > I found the answer. On the SeleniumFramework page is a pointer to a > worked example (see: http://www.mediawiki.org/wiki/ > SeleniumFramework#Working_example). The instructions for getting the > tests to work aren't totally transparent. The test file you include is: > > ../phase3/extensions/PagedTiffHandler/selenium/ PagedTiffHandler_tests.php > > (Not: ../phase3/extensions/PagedTiffHandler/tests/ > PagedTiffHandlerTest.php) > > Also, the instructions in SOURCES.txt specify getting all of the test > images from: > > http://www.libtiff.org/images.html > > But, when accessing the URL supplied on that page for the images (ftp:// > ftp.remotesensing.org/pub/libtiff/pics-3.6.1.tar.gz) a FILE NOT FOUND > error is returned. There is a new version of the pics file in ..libtiff, > but they do not contain the correct images. The correct URL is: ftp:// > ftp.remotesensing.org/pub/libtiff/old/pics-3.6.1.tar.gz. However, this > tar file does not include the images required by the PagedTiffHandler > tests. I thought I would apprise readers of this thread that I have the PagedTiffHandler example working. The problem turned out to be a bug in the Framework that meant LocalSeleniumSettings wasn't being read at the correct point in the startup sequence. According to Markus, he has fixed this bug and is now testing it. I presume he will let us know when the fix is committed. If you want to run the example before the fix is available, just make all your configuration settings to RunSeleniumTests.php. Dan -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Tue, 18 May 2010 19:27:38 +0200, Markus Glaser wrote: > Hi Dan, > > I had these error messages once when I used Firefox 3.6 for testing. > Until recently, Selenium did not support this browser. Apparently now > they do, but I did not have a chance to test this yet. So the solution > for me was to point Selenium to a Firefox 3.5. > > Cheers, > Markus Hi Markus, I thought it might be best to move this discussion off-line for a bit until we get the problems sorted out and then post the solution(s) to wikitech-l. This thread is getting fairly long and is getting into fairly complex issues. I tried emailing you at the address shown in your posts, but the email was returned as undeliverable. My email address is dness...@yahoo.com. If you think taking the issue off-line while we sort it out is a good thing to do, then email me from some address that I can use and I will update you on the status of my attempts to get PagedTiffHandler_tests.php to work. As a teaser, it appears there is a problem with the sequence of processing vis-a-vis LocalSettings and LocalSeleniumSettings Cheers, Dan -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Tue, 18 May 2010 19:27:38 +0200, Markus Glaser wrote: > Hi Dan, > > I had these error messages once when I used Firefox 3.6 for testing. > Until recently, Selenium did not support this browser. Apparently now > they do, but I did not have a chance to test this yet. So the solution > for me was to point Selenium to a Firefox 3.5. > > Cheers, > Markus My OS is Ubuntu 8.04. The version of Firefox is 3.0.19. Since Ubuntu automatically updates versions of its software, I assume this is the most up-to-date. Is there a list of browser versions compatible with selenium? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Tue, 18 May 2010 01:04:01 +0200, Markus Glaser wrote: > Hi Dan, > > the test fails at checking the prerequisites. It tries to load the image > page and looks for a specific div element which is not present if the > image was not uploaded correctly (id=filetoc). This might have changed > across the versions of MediaWiki. > > Did you install the PagedTiffHandler extension? It depends on > ImageMagick, so it might have rejected the upload. Although then it > should have produced an error message ;) So the other question is, which > MediaWiki version do you run the tests on? > > Regards, > Markus Hi Markus, Some further information. I originally uploaded Multipage.tiff before the extension was installed. I thought this might be the problem, so I uploaded it again after the extension was available. However, this did not solve the problem. Also, I am getting the following error from the selenium-server: 08:41:10.344 INFO - Got result: ERROR Server Exception: sessionId led to start new browser session: org.openqa.selenium.server.browserlaunchers.InvalidBrowserExecutableException: The specified path to the browser executable is invalid. doesn't exist; perhaps this session was already stopped? on session led to start new browser session: org.openqa.selenium.server.browserlaunchers.InvalidBrowserExecutableException: The specified path to the browser executable is invalid. 08:41:10.347 INFO - Command request: testComplete[, ] on session led to start new browser session: org.openqa.selenium.server.browserlaunchers.InvalidBrowserExecutableException: The specified path to the browser executable is invalid. 08:41:10.347 INFO - Got result: OK on session led to start new browser session: org.openqa.selenium.server.browserlaunchers.InvalidBrowserExecutableException: The specified path to the browser executable is invalid. I am using a LocalSeleniumSettings.php with the following parameters: // Hostname of selenium server $wgSeleniumTestsSeleniumHost = 'http://localhost'; // URL of the wiki to be tested. $wgSeleniumTestsWikiUrl = 'http://localhost'; // Wiki login. Used by Selenium to log onto the wiki $wgSeleniumTestsWikiUser = 'Wikiadmin'; $wgSeleniumTestsWikiPassword = 'Wikiadminpw'; // Use the *chrome handler in order to be able to test file uploads $wgSeleniumTestsBrowsers['firefox'] = '*firefox /usr/bin/firefox'; $wgSeleniumTestsBrowsers['ff-chrome'] = '*chrome /usr/bin/firefox'; // Actually, use this browser $wgSeleniumTestsUseBrowser = 'ff-chrome'; Regards, -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Tue, 18 May 2010 01:04:01 +0200, Markus Glaser wrote:
> Hi Dan,
>
> the test fails at checking the prerequisites. It tries to load the image
> page and looks for a specific div element which is not present if the
> image was not uploaded correctly (id=filetoc). This might have changed
> across the versions of MediaWiki.
>
> Did you install the PagedTiffHandler extension? It depends on
> ImageMagick, so it might have rejected the upload. Although then it
> should have produced an error message ;) So the other question is, which
> MediaWiki version do you run the tests on?
>
> Regards,
> Markus
Hi Markus,
I am running on the latest version in trunk (1.17alpha r66296). There was
no error when I uploaded the image. All of the extended details seem
correct. I installed the extension. I don't have either exiv2 or vips
installed, but according to the installation instructions these are
optional.
Here are the configuration values I used:
# PagedTiffHandler extension
require_once("$IP/extensions/PagedTiffHandler/PagedTiffHandler.php");
$wgTiffIdentifyRejectMessages = array(
'/^identify: Compression algorithm does not support random
access/',
'/^identify: Old-style LZW codes, convert file/',
'/^identify: Sorry, requested compression method is not
configured/',
'/^identify: ThunderDecode: Not enough data at scanline/',
'/^identify: .+?: Read error on strip/',
'/^identify: .+?: Can not read TIFF directory/',
'/^identify: Not a TIFF/',
);
$wgTiffIdentifyBypassMessages = array(
'/^identify: .*TIFFReadDirectory/',
'/^identify: .+?: unknown field with tag .+? encountered/'
);
$wgImageMagickIdentifyCommand = '/usr/bin/identify';
$wgTiffUseExiv = false;
$wgTiffUseVips = false;
// Maximum number of embedded files in tiff image
$wgTiffMaxEmbedFiles = 1;
// Maximum resolution of embedded images (product of width x height
pixels)
$wgTiffMaxEmbedFileResolution = 2560; // max. Resolution 1600 x 1600
pixels
// Maximum size of meta data
$wgTiffMaxMetaSize = 67108864; // 64kB
// TTL of Cacheentries for Errors
$wgTiffErrorCacheTTL = 84600;
Is there some way to use the wiki to look for the file property that is
causing the problem?
Regards,
Dan
--
-- Dan Nessett
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Mon, 17 May 2010 22:54:38 +0200, Markus Glaser wrote: > Hi Dan, > > will provide a working example with no need to include any extensions in > the course of this week. In the meantime, you might want to make sure > that $wgSeleniumTiffTestUploads = false; > in PagedTiffHandler_tests.php. Then, the test will not try to upload any > of the pictures from libtiff. In order for the tests to succeed, you > need to upload Multipage.tiff into the wiki. If there are any images > missing, please let me know and I will send them to you. Actually, I > didn't want to check in a third-party archive into the svn because of > copyright considerations. The images seem to be public domain, but to > me, it was not totally clear, whether they are. Are there any policies > regarding this case? I assume, when there are more tests, especially > with file uploads, the issue might arise again. > > Cheers, > Markus Thanks Markus, $wgSeleniumTiffTestUploads does indeed equal false. I was failing on the upload of Multipage.tiff until I added 'tiff' to $wgFileExtensions. Now I am failing because allChecksOk is false. It appears this happens on line 32 of PagedTiffHandler_tests.php on the statement: if ($source != 'filetoc') $this->allChecksOk = false; I'm not an image expert, so I don't know why this is happening. Regards, Dan -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Mon, 17 May 2010 19:11:21 +, Dan Nessett wrote: > During the meeting last Friday, someone (I sorry, I don't remember who) > mentioned he had created a test that runs with the currently checked in > selenium code. Is that test code available somewhere (it doesn't appear > to be in the current revision)? I found the answer. On the SeleniumFramework page is a pointer to a worked example (see: http://www.mediawiki.org/wiki/ SeleniumFramework#Working_example). The instructions for getting the tests to work aren't totally transparent. The test file you include is: ../phase3/extensions/PagedTiffHandler/selenium/PagedTiffHandler_tests.php (Not: ../phase3/extensions/PagedTiffHandler/tests/ PagedTiffHandlerTest.php) Also, the instructions in SOURCES.txt specify getting all of the test images from: http://www.libtiff.org/images.html But, when accessing the URL supplied on that page for the images (ftp:// ftp.remotesensing.org/pub/libtiff/pics-3.6.1.tar.gz) a FILE NOT FOUND error is returned. There is a new version of the pics file in ..libtiff, but they do not contain the correct images. The correct URL is: ftp:// ftp.remotesensing.org/pub/libtiff/old/pics-3.6.1.tar.gz. However, this tar file does not include the images required by the PagedTiffHandler tests. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
During the meeting last Friday, someone (I sorry, I don't remember who) mentioned he had created a test that runs with the currently checked in selenium code. Is that test code available somewhere (it doesn't appear to be in the current revision)? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
One of the URLs supplied by Ryan during the recent phone conference doesn't work. Specifically: http:// grid.tesla.usability.wikimedia.org:. I get the error: HTTP ERROR: 404 NOT_FOUND RequestURI=/ ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A potential land mine
--- On Mon, 8/24/09, Alex wrote: > I don't > believe anyone > except you has actually proposed restructuring the > extensions directory. Perhaps not. But, I don't see why that is relevant. I am making arguments why the extensions directory should be restructured. I may convince no one, but I don't think I should presume that. > A) There > aren't that many extensions that add command line utilities > (several > extensions also have scripts and hook based extensions so > wouldn't > neatly fit into such categories) Here are the files in /extensions/ that reference /maintenance/command.inc. There are 65 of them (line number of the reference at the end). I don't know which of these are commonly used and therefore included in installation extension/ directories, but I assume all of them are used by at least a small number of sites (otherwise, why include them in the extensions directory at all?) /extensions/AbuseFilter/install.php:8 /extensions/AbuseFilter/phpTest.php:8 /extensions/AdvancedSearch/populateCategorySearch.php:9 /extensions/AntiSpoof/batchAntiSpoof.php:6 /extensions/AntiSpoof/generateEquivset.php:4 /extensions/Babel/txt2cdb.php:9 /extensions/BoardVote/voterList.php:6 /extensions/CentralAuth/migratePass0.php:8 /extensions/CentralAuth/migratePass1.php:8 /extensions/CentralAuth/migrateStewards.php:3 /extensions/CentralNotice/rebuildLocalTemplates.php:3 /extensions/CentralNotice/rebuildTemplates.php:3 /extensions/CheckUser/importLog.php:4 /extensions/CheckUser/install.php:8 /extensions/cldr/rebuild.php:11 /extensions/CodeReview/svnImport.php:6 /extensions/CommunityVoice/CLI/Initialize.php:4 /extensions/Configure/findSettings.php:18 /extensions/Configure/manage.php:19 /extensions/Configure/migrateFiles.php:17 /extensions/Configure/migrateToDB.php:16 /extensions/Configure/writePHP.php:18 /extensions/DataCenter/CLI/Import.php:4 /extensions/DataCenter/CLI/Initialize.php:4 /extensions/DumpHTML/dumpHTML.php:61 /extensions/DumpHTML/wm-scripts/old/filterNamespaces.php:4 /extensions/DumpHTML/wm-scripts/queueController.php:6 /extensions/FlaggedRevs/maintenance/clearCachedText.php:13 /extensions/FlaggedRevs/maintenance/reviewAllPages.php:8 /extensions/FlaggedRevs/maintenance/updateAutoPromote.php:8 /extensions/FlaggedRevs/maintenance/updateLinks.php:10 /extensions/FlaggedRevs/maintenance/updateQueryCache.php:8 /extensions/FlaggedRevs/maintenance/updateStats.php:8 /extensions/LiquidThreads/compat/generateCompatibilityLocalisation.php:6 /extensions/LiquidThreads/import/import-parsed-discussions.php:4 /extensions/LiquidThreads/migrateDatabase.php:7 /extensions/LocalisationUpdate/update.php:7 /extensions/MetavidWiki/maintenance/download_from_archive_org.php:4 /extensions/MetavidWiki/maintenance/maintenance_util.inc.php:15 /extensions/MetavidWiki/maintenance/metavid2mvWiki.inc.php:16 /extensions/MetavidWiki/maintenance/metavid_gov_templates.php:2 /extensions/MetavidWiki/maintenance/mv_oneTime_fixes.php:2 /extensions/MetavidWiki/maintenance/mv_update.php:6 /extensions/MetavidWiki/maintenance/ogg_thumb_insert.php:15 /extensions/MetavidWiki/maintenance/scrape_and_insert.inc.php:12 /extensions/MetavidWiki/maintenance/transcode_to_flv.php:13 /extensions/MetavidWiki/maintenance/video_ocr_thumb_insert.php:15 /extensions/OAI/oaiUpdate.php:17 /extensions/ParserFunctions/testExpr.php:4 /extensions/SecurePoll/voterList.php:11 /extensions/SemanticMediaWiki/maintenance/SMW_conceptCache.php:18 /extensions/SemanticMediaWiki/maintenance/SMW_dumpRDF.php:34 /extensions/SemanticMediaWiki/maintenance/SMW_refreshData.php:41 /extensions/SemanticMediaWiki/maintenance/SMW_setup.php:46 /extensions/SemanticMediaWiki/maintenance/SMW_unifyProperties.php:27 /extensions/SemanticResultFormats/Ploticus/SRF_Ploticus_cleanCache.php:24 /extensions/SemanticTasks/ST_CheckForReminders.php:6 /extensions/SpamBlacklist/cleanup.php:9 /extensions/SwarmExport/swarmExport.php:23 /extensions/TitleKey/rebuildTitleKeys.php:3 /extensions/TorBlock/loadExitNodes.php:7 /extensions/TrustedXFF/generate.php:8 /extensions/UsabilityInitiative/PrefStats/populatePrefStats.php:9 /extensions/WikiAtHome/internalCmdLineEncoder.php:6 /extensions/WikiTrust/sql/create_db.php:74 Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A potential land mine
--- On Mon, 8/24/09, Chad wrote: > Why skip trying to find the location? > If MW_INSTALL_PATH > is already missing, what have we got to lose from trying > to guess the location? The vast majority of people don't > screw with the default structure, so it should be just > fine. That's a reasonable question, stating in another way the useful maxim, "if it ain't broke, don't fix it." The problem is I think it's "broke". Here is my take on the pros/cons of leaving things unchanged: Pros: * Some administrators are used to simply typing the line php .php. Making them type: MW_INSTALL_PATH=/var/wiki/mediawiki php .php would be inconvenient. In answer to this, for the MW installations running on unix, it is pretty simple to alias "MW_INSTALL_PATH=/var/wiki/mediawiki php" and put the definition into .bash_profile (or the appropriate shell initialization script). This is a one time effort and so the change isn't as onerous as it might seem. I assume there is a similar tactic available for windows systems. Cons: * The use of file position dependent code is a problem during development and much less of a problem during installation and production (as you suggest). Right now there are ~400 sub-directories in the extensions directory. It seems to me reorganization of the extensions directory would help understanding the relationship between individual extensions and the core. For example, having two subdirectories, one for cli utilities and another for hook based extensions would clarify the role each extension plays. However, currently there are 29 extensions where $IP is set using the relative position of the file in the MW directory structure (a couple of other extensions set $IP based on MW_INSTALL_PATH). Reorganizing the directory structure has the potential of breaking them. * CLI utilities are moved around for reasons other than a reorganization of the extensions directory. For example, as I understand it, DumpHTML was moved from maintenance/ to extensions/. dumpHTML.php sets $IP based on its relative position in the distribution tree. It was a happy coincidence that when it was moved, its relative position didn't change. However, it is unreasonable to think such reclassifications will always be as fortunate. Since the cons outweigh the pros, I remain convinced that the change I suggested (using die()) improves the code. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A potential land mine
--- On Sun, 8/23/09, Aryeh Gregor wrote:
> If they can run commands on the command line, then they can
> use
> environment variables. If they can't, then your
> suggestion doesn't
> help.
>
> > If there are administrators who can execute command
> lines, but cannot set environmental variables (e.g., they
> are confined to use a special shell)
>
> There aren't. That would make no sense.
Thanks for clarifying the situation. Given this information I suggest changing
all code in command line utilities of the form:
$IP = getenv( 'MW_INSTALL_PATH' );
if ( $IP === false ) {
$IP = dirname(__FILE__).'/../..';
}
to:
$IP = getenv( 'MW_INSTALL_PATH' );
if ( $IP === false ) {
echo "Error. The environmental variable MW_INSTALL_PATH must be set to the root
of the MW distribution. Exiting.\n";
die();
}
This would eliminate file position dependent code from the command line
utilities, making them easier to maintain (i.e., they can be moved in the
distribution without breaking them).
Dan
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A potential land mine
--- On Sun, 8/23/09, dan nessett wrote: In my last email, I quoted Andrew Garret: > $ MW_INSTALL_PATH=/var/wiki/mediawiki php/maintenance/update.php This was incorrect. I fumbled some of the editing in my reply. What he proposed was: > $ MW_INSTALL_PATH=/var/wiki/mediawiki php maintenance/update.php Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A potential land mine
--- On Sun, 8/23/09, Andrew Garrett wrote:
> $ MW_INSTALL_PATH=/var/wiki/mediawiki php/maintenance/update.php
I don't understand the point you are making. If an MW administrator can set
environmental variables, then, of course, what you suggests works. However,
Brion mentions in his Tues, Aug 11, 10:09 email that not every MW installation
admin can set environmental variables and Aryeh states in his Tues, Aug. 11,
10:09am message that some MW administrators only have FTP access to the
installations they manage. So, as I understand it some administrators cannot
use the tactic you describe.
An important issue is whether these admins have access to command line
utilities at all. If not, then the use of file position dependent code in
command line utilities can be eliminated by substituting:
$IP = getenv( 'MW_INSTALL_PATH' );
if ( $IP === false ) die();
for (taken from dumpHTML.php):
$IP = getenv( 'MW_INSTALL_PATH' );
if ( $IP === false ) {
$IP = dirname(__FILE__).'/../..';
This works if only admins who can set environmental variables can execute MW
command line utilities.
If there are administrators who can execute command lines, but cannot set
environmental variables (e.g., they are confined to use a special shell), then
what I suggested in the previous email eliminates file position dependency.
That is, the command line would be:
php -d include_path=:
.php
If an admin can execute php .php, he should be able to execute the
prior command.
Dan
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A potential land mine
--- On Sat, 8/22/09, Platonides wrote: > How's that better than MW_CONFIG_PATH environment > variable? My understanding is that the administrators of certain installations cannot set environmental variables (I am taking this on faith, because that seems like a very very restricted site). What I suggested takes no work at all by installers or wiki administrators. MWInit.php (or whatever people want to name it) is part of the distribution. When you run a command line utility you have to be able to type something like "php .php". If you can do that, you should be able to type "php -d .php". If I am wrong and for some sites even that is not possible, then I am not sure how you would use command line utlities at all. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A potential land mine
--- On Fri, 8/21/09, Andrew Garrett wrote: > Yes, this is where we started because this is the status > quo. What I > was describing is how it's done now. Is maintaining the status quo really desirable? Look at the extensions directory. It currently has ~400 extension sub-directories. If you wanted to reorganize this so there is some sort of logic to it (e.g., first cut - put command line extensions in one directory and hook based extensions in another) you would have to change a lot of code so the computation of $IP by some extensions is correct. How about this. When you run php from the command line, there is a flag -d defined as: "foo[=bar] Define INI entry foo with value 'bar'". For those extensions that run from the command line you could require callers to include a value for the php "include_path". This value would be the value in php.ini appended with the value of a directory containing the simple MWInit.php file provided in my message of Aug. 11, 9:57AM. Command line extensions could then call MWInit() to get MW root. I just tried this and it worked. It would fix the problem for at least command line utilities. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] CPRT feasibility
--- On Thu, 8/20/09, Andrew Garrett wrote: > As the title implies, it is a performance limit report. You > can remove > it by changing the parser options passed to the parser. > Look at the > ParserOptions and Parser classes. Thanks. It appears dumpHTML has no command option to turn off this report (the parser option is mEnableLimitReport). A question to the developer community: Is it better to change dumpHTML to accept a new option (to turn off Limit Reports) or copy dumpHTML into a new CPRT extension and change it. I strongly feel that having two extensions with essentially the same functionality is bad practice. On the other hand, changing dumpHTML means it becomes dual purposed, which has the potential of making it big and ugly. One compromise position is to attempt to factor dumpHTML so that a core provides common functionality to two different upper layers. However, I don't know if that is acceptable practice for extensions. A short term fix is to pipe the output of dumpHTML through a filter that removes the Limit Report. That would allow developers to use dumpHTML (as a CPRT) fairly quickly to find and fix the known-to-fail parser bugs. The downside to this is it may significantly degrade performance. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] CPRT feasibility
I am looking into the feasibility of writing a comprehensive parser regression test (CPRT). Before writing code, I thought I would try to get some idea of how well such a tool would perform and what gotchas might pop up. An easy first step is to run dump_HTML and capture some data and statistics. I tried to run the version of dumpHTML in r54724, but it failed. So, I went back to 1.14 and ran that version against a small personal wiki database I have. I did this to get an idea of what structures dump_HTML produces and also to get some performance data with which to do projections of runtime/resource usage. I ran dumpHTML twice using the same MW version and same database. I then diff'd the two directories produced. One would expect no differences, but that expectation is wrong. I got a bunch of diffs of the following form (I have put a newline between the two file names to shorten the line length): diff -r HTML_Dump/articles/d/n/e/User~Dnessett_Bref_Examples_Example1_Chapter_1_4083.html HTML_Dump2/articles/d/n/e/User~Dnessett_Bref_Examples_Example1_Chapter_1_4083.html 77,78c77,78 < Post-expand include size: 16145/2097152 bytes < Template argument size: 12139/2097152 bytes --- > Post-expand include size: 16235/2097152 bytes > Template argument size: 12151/2097152 bytes I looked at one of the html files to see where these differences appear. They occur in an html comment: Does anyone have an idea of what this is for? Is there any way to configure MW so it isn't produced? I will post some performance data later. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] identifier collisions
--- On Mon, 8/17/09, Brion Vibber wrote: > Really though, this thread has gotten extremely unfocused; > it's not > clear what's being proposed to begin with and we've > wandered off to a > lot of confusion. I'll take partial responsibility for the confusion. Like I said recently, I think it is pretty ridiculous that a newbie like me is pushing the discussion on MW QA. I am trying to learn the underlying technologies as fast as I can, but it is a steep learning curve. Also, let me reiterate. If someone more knowledgeable about these technologies than I is willing to step up and lead, I have no problem whatsoever following. On the other hand, I need a MW product regression test suite. If getting it means I have to expose my ignorance to an international audience, I'm willing to take that hit. > We should probably start up a 'testing center' page on > mediawiki.org > and begin by summarizing what we've already got -- then we > can move on > to figuring out what else we need. > I think this is a great idea. > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] identifier collisions
--- On Fri, 8/14/09, Tim Starling wrote: > And please, spare us from your rant about how terrible this > is. It's > not PHP's fault that you don't know anything about it. I'm sorry my questions make you angry. I don't recall ranting about PHP. Actually, I kind of like it. Lack of thread safety is an implementation problem not a problem with the language. But, let's not dwell on your rant about my stupidity. Let's do something positive. You are an expert on the MW software and presumably PHP, Apache and MySQL. If you find it ridiculous that a newbie is driving the discussion about MW QA (I certainly do), pick up the ball and run with it. How would you fix the parser so all disabled tests in parserTests run? How would you build a test harness so developers can write unit tests for their bug fixes, feature additions and extensions? How would you integrate these unit tests into a good set of regression tests? How would you divide up the work so a set of developers can complete it in a reasonable amount of time? How do you propose achieving consensus on all of this? On the other hand, maybe you would rather code than think strategically. Fine. Commit yourself to fixing the parser so all of the disabled tests run and also all or most of the pages on Wikipedia do not break and I will shut up about the CPRT. Commit yourself to creating a test harness that other developers can use to write unit tests and I will gladly stop writing emails about it. Commit yourself to develop the software the organizes the unit tests into a product regression test that developers can easily run and I will no longer bother you about MW QA. My objective is a MW regression test suite that provides evidence that any extensions I write do not break the product. Once that objective is achieved, I will no longer infect your ears with dumb questions. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] identifier collisions
--- On Fri, 8/14/09, Dmitriy Sintsov wrote: > I remember some time ago I was strongly discouraged to > compile and run > PHP threaded MPM for apache because some functions or > libraries of PHP > itself were not thread safe. While my machine was compiling AMP components, I thought about this a little bit. It seems weird that the implementation of a language intended to provide backend functionality for web servers isn't thread safe. Apache and other web server software must be threaded operationally towards infinity. How do they deal with using a non-thread safe library? Each time a thread executes PHP code does it have to grab a lock that protects the whole PHP library? Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] identifier collisions
--- On Fri, 8/14/09, Dmitriy Sintsov wrote: > I remember some time ago I was strongly discouraged to > compile and run > PHP threaded MPM for apache because some functions or > libraries of PHP > itself were not thread safe. OK, this and Chad's comment suggests the option is multi-process/IPC. One more question: * Can we assume PHP has PCNLT support or will the test require startup from a shell script? Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] identifier collisions
One of the first problems to solve in developing the proposed CPRT is how to call a function with the same name in two different MW distributions. I can think of 3 ways: 1) use the Namespace facility of PHP 5.3, 2) use threads, or 3) use separate process and IPC. Since MAMP supports none of these I am off building an AMP installation from scratch. Some questions: * Are there other ways to solve the identifier collision problem? * Are some of the options I mention unsuitable for a MW CPRT, e.g., currently MW only assumes PHP 5.0 and requiring 5.3 may unacceptably constrain the user base. * Is MW thread safe? Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A comprehensive parser regression test
--- On Wed, 8/12/09, Tim Landscheidt wrote: > I think though that more people > would read and embrace your > thoughts if you would find > a more concise way to put > them across :-). Mea Culpa. I'll shut up for a while. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] More fun and games with file position relative code
--- On Wed, 8/12/09, Brion Vibber wrote: > Your setup is incorrect: the extensions folder *always* > goes inside the > MediaWiki root dir. Always. > Sorry, my inexperience with Subversion led me in the wrong direction. I didn't realize I could check out phase3 then point Subversion to the extensions directory in phase3 to check out extensions. I thought Subversion would get confused, but I just tried it and it worked :-). At least Subversion performed the checkout. I haven't tried doing an update. I just (case sensitively) searched the extensions directory for the string "$IP =" and found 32 files that compute $IP on their own. How about creating a standard bit of code that extensions and other modules can copy and use to figure out MW root. For example, it is very unlikely that the name of the first level directory (i.e., phase3) will change. The code can call dirname( __FILE__ ) and then search from the end of the pathname until it finds phase3. It then knows that the prefix is MW root. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] More fun and games with file position relative code
Chad wrote: > DumpHTML will not be moved back to maintenance in the repo, it was > already removed from maintenance and made into an extension. Issues > with it as an extension should be fixed, but it should not be encouraged > to go back into core. What I meant was I can move the code in DumpHTML as CPRT to maintanence in my working copy and work on it there. Whether this code is simply a MacGyver test or something else is completely up in the air. > Also, on a meta notecan you possibly confine all of your testing comments > to a single thread? We don't need a new thread for each problem you find :) > My apologies. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A comprehensive parser regression test
--- On Wed, 8/12/09, Roan Kattouw wrote: > I read this paragraph first, then read the paragraph above > and > couldn't help saying "WHAT?!?". Using a huge set of pages > is a poor > replacement for decent tests. I am not proposing that the CPRT be a substitute for "decent tests." We still need a a good set of tests for the whole MW product (not just the parser). Nor would I recommend making a change to the parser and then immediately running the CPRT. Any developer that isn't masochistic would first run the existing parserTests and ensure it passes. Then, you probably want to run the modified DumpHTML against a small random selection of pages in the WP DB. Only if it passes those tests would you then run the CPRT for final assurance. The CPRT I am proposing is about as good a test of the parser that I can think of. If a change to the parser passes it using the Wikipedia database (currently 5 GB), then I would say for all practical purposes the changes made to the parser do not regress it. > Also, how would you handle > intentional > changes to the parser output, especially when they're > non-trivial? I don't understand this point. Would you elaborate? Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] More fun and games with file position relative code
So. I checked out a copy of phase3 and extensions to start working on
investigating the feasibility of a comprehensive parser regression test. After
getting the working copy downloaded, I do what I usually do - blow away the
extensions directory stub that comes with phase3 and soft link the downloaded
copy of extensions in its place. I then began familiarizing myself with
DumpHTML by starting it up in a debugger. Guess what happened.
If fell over. Why? Because DumpHTML is yet another software module that
computes the value $IP. So what? Well, DumpHTML.php is located in
../extensions/DumpHTML. At line 57-59 it executes:
$IP = getenv( 'MW_INSTALL_PATH' );
if ( $IP === false ) {
$IP = dirname(__FILE__).'/../..';
}
This works on a deployed version of MW, since the extensions directory is
embedded in /phase3. But, in a development version, where /extensions is a
separate subdirectory, "./../.." does not get you to phase3, it gets you to MW
root. So, when you execute the next line:
require_once( $IP."/maintenance/commandLine.inc" );
DumpHTML fails.
Of course, since I am going to change DumpHTML anyway, I can move it to
/phase3/maintenance and change the '/../..' to '/..' and get on with it. But,
for someone attempting to fix bugs in DumpHTML, the code that uses a knowledge
of where DumpHTML.php is in the distribution tree is an issue.
Dan
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A comprehensive parser regression test
--- On Wed, 8/12/09, dan nessett wrote: > "If you ran this test on, for example, Wikipedia, Of course, what I meant is run the test on the Wikipedia database, not on the live system. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] A comprehensive parser regression test
I am investigating how to write a comprehensive parser regression test. What I mean by this is something you wouldn't normally run frequently, but rather something that we could use to get past the "known to fail" tests now disabled. The problem is no one understands the parser well enough to have confidence that if you fix one of these tests that you will not break something else. So, I thought, how about using the guts of DumpHTML to create a comprehensive parser regression test. The idea is to have two versions of phase3 + extensions, one without the change you make to the parser to fix a known-to-fail test (call this Base) and one with the change (call this Current). Modify DumpHTML to first visit a page through Base, saving the HTML then visit the same page through Current and compare the two results. Do this for every page in the database. If there are no differences, the change in Current works. Sitting here I can see the eyeballs of various developers bulging from their faces. "What?" they say. "If you ran this test on, for example, Wikipedia, it could take days to complete." Well, that is one of the things I want to find out. The key to making this test useful is getting the code in the loop (rendering the page twice and testing the results for equality) very efficient. I may not have the skills to do this, but I can at least develop an upper bound on the time it would take to run such a test. A comprehensive parser regression test would be valuable for: * fixing the known-to-fail tests. * testing any new parser that some courageous developer decides to code. * testing major releases before they are released. * catching bugs that aren't found by the current parserTest tests. * other things I haven't thought of. Of course, you wouldn't run this thing nightly or, perhaps, even weekly. Maybe once a month would be enough to ensure the parser hasn't regressed out of sight. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Assumptions for development machines (w.r.t. to test infrastructure)
--- On Wed, 8/12/09, Roan Kattouw wrote: > On shared hosting, both are impossible. MediaWiki currently > works with > minimal write access requirements (only the config/ > directory for the > installer and the images/ directory if you want uploads), > and we'd > like to keep it that way for people who are condemned to > shared > hosting. Which is why I wrote in the message that is the subject of your reply: "I gave up on the proposal when people pointed out that MW admins may not have the privileges that would allow the install script to do either." Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Assumptions for development machines (w.r.t. to test infrastructure)
--- On Wed, 8/12/09, Brion Vibber wrote: > > The suggestions were for explicit manual configuration, not > > autodiscovery. Autodiscovery means *not* having to set > anything. :) I am insane to keep this going, but the proposal I made did not require doing anything manually (other than running the install script, which you have to do anyway). The install script knows (or can find out) where the MW root is located. It could then either: 1) rewrite php.ini to concatenate the location of MWInit.php at the end of include_path, or 2) plop MWInit.php into a directory already searched by PHP for includes/requires (e.g., the PEAR) directory. I gave up on the proposal when people pointed out that MW admins may not have the privileges that would allow the install script to do either. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Assumptions for development machines (w.r.t. to test infrastructure)
--- On Wed, 8/12/09, Brion Vibber wrote: > We *already* automatically discover the MW root directory. Yes, you're right. I should have said automatically discover the MW root directory without using file position dependent code. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Problem with phpunit --skeleton
I have been playing around with phpunit, in particular its facility for generating tests from existing PHP code. You do this by processing a suitably annotated (using /* @assert ... */ comment lines) version of the file with phpunit --skeleton. Unfortunately, the --skeleton option assumes the file contains a class with the same name as the file. For example, I tried annotating Hook.php and then processing it with phpunit --skeleton. It didn't work. phpunit reported: Could not find class "Hook" in ".../phase3/tests/Hook.php" (where I have replaced my path to MW with "..."). Since there are many MW files that do not contain classes of the same name as the file (or even classes at all), the --skeleton is probably not very useful for MW phpunit test generation. You can still use phpunit by inserting the appropriate test code manually, but the objective of automatically generating tests with the same convenience as adding lines to parserTests.txt isn't available. Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Assumptions for development machines (w.r.t. to test infrastructure)
--- On Wed, 8/12/09, Chad wrote: > > Tests should run in a vanilla install, with minimal > dependency on > external stuff. PHPUnit > (or whatever framework we use) would be considered an > acceptable dependency for > test suites. If PHPUnit isn't available (ie: already > installed and in > the include path), then > we should bail out nicely. > > In general, external dependencies should be used as > seamlessly as > possible, with minimal > involvement from the end-user. A good example is > wikidiff2...we load > it if it exists, we fail > back to PHP diff mode if we can't use it. OK, graceful backout if an external dependency fails and minimal dependency on external stuff. So far we have two categories of proposal for test infrastructure : 1) build it all ourselves, and 2) use some external stuff. How do we decide which to do? Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Assumptions for development machines (w.r.t. to test infrastructure)
I'm starting a new thread because I noticed my news reader has glued together messages with the title "A potential land mine" and "MW test infrastructure architecture," which may confuse someone coming into the discussion late. Also, the previous thread has branched into several topics and I want to concentrate on only one, specifically what can we assume about the system environment for a test infrastructure? These assumptions have direct impact on what test harness we use. Let me start by stating what I think can be assumed. Then people can tell me I am full of beans, add to the assumptions, subtract from them, etc. The first thing I would assume is that a development system is less constrained than a production system in what can and what cannot be installed. For example, people shot down my proposal to automatically discover the MW root directory because some production systems have administrators without root access, without the ability to load code into the PEAR directory, etc. Fair enough (although minimizing the number of places where $IP is computed is still important). However, if you are doing MW development, then I think this assumption is too stringent. You need to run the tests in /tests/PHPUnitTests, which in at least one case requires the use of $wgDBadminuser and $wgDBadminpassword, something a non-priviledged user would not be allowed to do. If a developer has more system privileges than a production admin, to what extent? Can we assume he has root access? If not, can we assume he can get someone who has to do things like install PHPUnit? Can we assume the availability of PERL or should we only assume PHP? Can we assume *AMP (e.g., LAMP, WAMP, MAMP, XAMP)? Can we assume PEAR? Can the developer install into PEAR? Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] MW test infrastructure architecture
--- On Tue, 8/11/09, Chad wrote: > > Neither of these need to be tested directly. If > AutoLoader breaks, > > then some other class won't load, and the tests for > that class will > > fail. If wfRunHooks() fails, then some hook won't > work, and any test > > of that hook will fail. > > I will pass on commenting about these for the moment because: > > I think what's needed for decent test usage for > MediaWiki is: > > > > 1) Some test suite is picked. PHPUnit is probably > fine, if it runs > > out of the box and doesn't need some extra module to > be installed. > > > > 2) The test suite is integrated into CodeReview with > nag e-mails for > > broken tests. > > > > 3) A moderate number of tests are written for the test > suite. > > Existing parser tests could be autoconverted, > possibly. Maybe someone > > paid could be assigned to spend a day or two on this. > > > > 4) A new policy requires that everyone write tests for > all their bug > > fixes and enhancements. Commits that don't add > enough tests will be > > flagged as fixme, and reverted if not fixed. > > > > (4) is critical here. All good stuff, especially (4) - applause :-D. > > While we're at it, it would be nice if we instituted > some other > > iron-clad policies. Here's a proposal: > > > > * All new functions (including private helper > functions, functions in > > JavaScript includes, whatever) must have > function-level documentation > > that explains the purpose of the function and > describes its > > parameters. The documentation must be enough that no > MediaWiki > > developer should ever have to read the function's > source code to be > > able to use it correctly. Exception: if a method is > overridden which > > is already documented in the base class, it doesn't > need to be > > documented again in the derived class, since the > semantics should be > > the same. > > * All new classes must have high-level documentation > that explains > > their purpose and structure, and how you should use > them. The > > documentation must be sufficient that any MediaWiki > developer could > > understand why they might want to use the class in > another file, and > > how they could do so, without reading any of the > source code. Of > > course, developers would still have to read the > function documentation > > to learn how to use specific functions. There are no > exceptions, but > > a derived class might only need very brief > documentation. > > * All new config variables must have documentation > explaining what > > they do in terms understandable to end-users. They > must describe what > > values are accepted, and if the values are complicated > (like > > associative arrays), must provide at least one example > that can be > > copy-pasted. There are no exceptions. > > * If any code is changed so as to make a comment > incorrect, the > > comment must be updated to match. There are no > exceptions. > > > > Or whatever. We have *way* too few high-level > comments in our code. > > We have entire files -- added quite recently, mind > you, by established > > developers -- that have no or almost no documentation > on either the > > class or function level. We can really do better > than this! If we > > had a clear list of requirements for comments in new > code, we could > > start fixme'ing commits that don't have adequate > comments. I think > > that would be enough to get people to add sufficient > comments, for the > > most part. Without clear rules, though, backed up by > the threat of > > reverting, I don't think the situation will improve > here. Wonderful stuff - more applause. > > (Wow, this kind of turned into a thread hijack. :D) > > Who cares. It needs to be said. > > On the whole "new code" front. Can we all /please/ remember > that > we're writing PHP5 here. Visibility on all new functions > and variables > should also be a must. > OK, I must admit I didn't understand that, probably because I'm new to PHP. Can you make this more explicit? Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] MW test infrastructure architecture
--- On Tue, 8/11/09, Alexandre Emsenhuber wrote: > My idea is the move the "backend" of ParserTest > (parserTests.txt file > processing, result reporting, ...) and the TestRecorder > stuff to > something like a MediaWikiTests class that extends > Maintenance and > move the rest in a file in /maintenance/tests/ (to be > created) and re- > use the backend to have files that have the same format, > but test's > input could be raw PHP code (a bit like PHP core's tests) > with a new > config variable that's like $wgParserTestFiles but for > these kind of > test. This mostly concerns the actual tests in /tests/ and > /t/inc/). > We can also port cdb-test.php, fuzz-tester.php, > preprocessorFuzzTest.php and syntaxChecker.php to this new > system and > then create a script in /maintenance/ that runs all the > tests in / > maintenance/tests/. This allows to also upload all the > tests to > CodeReview, not only the parser tests. A benefit is that we > can get > ride of /tests/ and /t/. One of the beauties of open source code development is he who does the work wins the prize. Of course, I am sure senior developers have discretionary power on what goes into a release and what does not. But, if you want to do the work, go for it (says the guy [me] who just joined the group). However, I think you should consider the following: * parserTests is designed to test parsing, which is predominantly a text manipulation task. Other parts of MW do not necessarily provide text processing markers that can be used to decide whether they are working correctly or not. * Sometimes testing the action of a module requires determining whether a series of actions provide the correct behavior. As far as I am aware, parserTests has no facility to tie together a set of actions into a single test. For example, consider two MW files in phase3/includes: 1) AutoLoader.php and 2) Hooks.php. In AutoLoader, the method loadAllExtensions() loads all extensions specified in $wgAutoloadClasses. It takes no parameters and has no return value. It simply walks through the entries specified in $wgAutoloadClasses and if the class specified as the key exists, executes a require of the file specified in the value. I don't see how you would specify a test of this method using the syntax of parserTests.txt. In Hooks.php, there is a single function wfRunHooks(). It looks up hooks previously set and calls user code for them. It throws exceptions in certain error conditions and testing it requires setting a hook and seeing if it is called appropriately. I don't see how you could describe this behavior with parserTests.txt syntax. Of course, you could create new syntax and behavior for the parserTests software components, but that is a lot of work that other infrastructure has already done. For example, see the set of assertions for phpunit (http://www.phpunit.de/manual/2.3/en/api.html#api.assert.tables.assertions). Dan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
