FYI, after some consultation with Chris, we decided to implement
Mediawiki API setup steps in the MobileFrontEnd repo first, so all the
bits now live in this commit[1]
As things stand now, only wiki articles are being created via the API
before each test Scenario is run, but code for user account creation is
part of that commit as well (although that method doesn't currently get
invoked).
So if this solution seems workable to all concerned, it will accomplish
the goal of allowing each browser test to create whatever wiki articles
it needs to satisfy test assertions, and we can easily expand the scope
later to also allow each browser test to create user logins as needed.
- Jeff
[1] https://gerrit.wikimedia.org/r/#/c/106548/
On 1/7/14, 8:26 AM, Chris McMahon wrote:
I'm Ccing the QA mail list, this might be the place for this discussion.
What you're talking about is often called "hermetic" test
environments. Where at run time a test environment is created
sufficient to support the tests being run. Right now we're thinking
about this in two different ways:
One is, as you mentioned, creating a set of tests that are intended to
run in a bare wiki with no test data. We're identifying these tests
now in order to create a PhantomJS smoke test suite.
We are also exploring using the Mediawiki API to create or update
pages and users at run time to avoid the overhead of doing it through
the UI. I've had Jeff Hall start looking into this:
https://gerrit.wikimedia.org/r/#/c/105835/ .
Another avenue we have discussed is to use Vagrant. I haven't gone
very far down that path, but it should be possible to share a Vagrant
image with essential test data on it.
As to why we don't have this today, my first priority was to create a
viable *shared* test environment with shared test data. We needed
beta labs and test2wiki working properly to support fast deployments
and to avoid the "works on my machine" problems. Now that beta and
test2 are functioning reasonably well, we can turn attention to the
hermetic test environments.
-Chris
On Mon, Jan 6, 2014 at 6:43 PM, Jon Robson <[email protected]
<mailto:[email protected]>> wrote:
The best way to get round this would be to setup a new database every
time the tests begin and remove it at the end of the tests.
Somewhat related to
https://bugzilla.wikimedia.org/show_bug.cgi?id=56057
Chris any ideas?
On Mon, Jan 6, 2014 at 5:33 PM, Ryan Kaldari
<[email protected] <mailto:[email protected]>> wrote:
> Right now, we have a lot of important features that don't have any
> acceptance tests because in order to test them you have to
create a new
> account. This would result in thousands of bogus accounts being
created on
> all the wikis we test on. Does anyone have any ideas about how
we could
> solve this problem?
>
> Ryan Kaldari
_______________________________________________
QA mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/qa
_______________________________________________
QA mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/qa
