On Thu, Jun 4, 2015 at 10:34 AM, Dan Duvall <[email protected]> wrote:
> In other words, there were 12 non-merge commits made yesterday to > MobileFrontend alone; that's around 9 hours of run time for the tests to > complete which, on a single instance, will have to be run in serial to > ensure the tests don't interfere with one another; that essentially > nullifies the more expedient feedback that you're hoping to gain. > > You could scale up the number of instances in the pool, but at that point > the ad hoc setup will be duplicating a large portion of our shared CI > infrastructure, which brings me to the next unanswered question: Who is > going to maintain this setup? I don't mean to be a naysayer here, but I > worry about the effects of implementing such a complex—and seemingly > volatile setup—without a clear understanding within Infrastructure or > Reading about its real value or long-term maintenance burden. Perhaps this > is misplaced anxiety, but this screams tech debt incarnate. > A separate instance (or full serialization) for each test seems like an enormous waste of resources. At worst, each test needs a separate database and can share the VM and run fully parallel; setting up a new database from a dump of a known good state should be in the seconds range. Ideally, most tests would be read-only and marked as such, and those could run on the same DB.
_______________________________________________ QA mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/qa
