The way I've dealt with this in the past (prior to working at the WMF)
was to create Git branches for automated tests that were associated with
specific upcoming release cycles. Continuous integration tools like
Jenkins make it easy to manage builds for multiple branches, so the
procedural overhead was small.
This worked really well where the functional test code lived in a
separate repository from the application code under test. It might be a
little more complicated to implement where the tests are part of the
application repo (per the WMF practice), but should still be doable.
That's my 2 cents!
- Jeff
On 2/5/14, 3:10 PM, Chris McMahon wrote:
Having browser tests in the repos of the extensions being tested is
Phase 0. {{status|done}}
Now we need to find a way to associate browser tests with a particular
version or versions in time of the extension being tested.
Then we we would be able to pull master branch of tests to target
beta, pull pre-release branch of tests to target test2wiki, pull
production branch of tests to target prod. This is on my list to
investigate in the next quarter. Suggestions are welcome.
-Chris
On Wed, Feb 5, 2014 at 2:42 PM, Jon Robson <[email protected]
<mailto:[email protected]>> wrote:
Anyone?
On Fri, Jan 31, 2014 at 3:41 PM, Jon Robson <[email protected]
<mailto:[email protected]>> wrote:
> I'm not sure if any of these have bugs but please advise on the best
> approach to deal with them.
>
> 1) Various tests fail because they rely on code that has been merged
> to master but not deployed. e.g. new tests for code that has been
> merged to master, will fail on enwiki until enwiki gets the new code
> deployed.
> Is there anyway we can run the tests from the master branch of the
> project they belong to, to stop this from happening?
>
>
> 2) Due to bugs some tests randomly fail e.g.
> https://bugzilla.wikimedia.org/show_bug.cgi?id=60338
> Is there anyway we can flag these different in the email?
>
> Note: I have actually started filtering out mails from
> test2.m.wikipedia.org <http://test2.m.wikipedia.org> and
en.m.wikipedia.org <http://en.m.wikipedia.org> because of point 1.
_______________________________________________
QA mailing list
[email protected] <mailto:[email protected]>
https://lists.wikimedia.org/mailman/listinfo/qa
_______________________________________________
QA mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/qa
_______________________________________________
QA mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/qa
