2008/3/4, Stepan Mishura <[EMAIL PROTECTED]>: > Hi folks, > > I've moved 2.0 build-and-test's branch to the trunk - the branch was > used during M2-M5 testing and IMO it proved to be very useful (also I > tagged the former trunk as 1.0.) >
Great news. Thanks. > I think that this is a good time to collect ideas and hear opinions > what should be improved and possible directions for test infra > development. > Shall we supply a friendly front-end for user? List all the suites available and make the configuration more easy. > I have to admit that the current testing infra is far from being > perfect �C there a lot for work to make is more usable, cleaning up > current scripts, sort out all know issues with tests/scenarios and so > on. But I'd like to address problems I believe of different level. I > have an impression that Harmony tests suites are still in shadow (IOW, > IMHO they are undervalued) of testing automation (i.e. CruiseControl > and support scripts) �C and most of folks prefer to contribute to the > code development. And I believe that it is possible to make tests > development also interesting too. IMO the missed point that most of Agree :) > Harmony test suites can be used as stand alone suites to verify/test > different implementations �C in other words, the suites are not tightly > coupled with DRL VM, for example, it is possible to run them against > IBM VME (BTW, may it be makes sense to run full set of suites against > IBM VME for each milestone candidate as for DRL VM. Any volunteers?) > It sounds reasonable, in fact, i am going to plan the same thing. Maybe we can make our BTI a more flexible infrastrcture for more runtime. We can pick up a runtime before starting the BTI as well as the test suite, such as: drlvm+harmony -| |- classlib unit test j9vm+harmony -| -- BTI ----|- vm unit test ... -| |- ... Volunteer :) > I'm going to update the test infra home page [1] with info related to > the renewed trunk. Also I think the page should contain some words > about project's testing strategy: i.e. how Harmony QA and release > process are organized - integrity/snapshot testing, milestones, code > freeze and etc., what suites are there (their description and > purpose). I think this may encourage people to invest efforts into > developing tests. > > Other ideas and thoughts? > > [1] http://harmony.apache.org/subcomponents/buildtest/index.html > > Thanks, > Stepan. > -- Sean, Xiao Xia Qiu China Software Development Lab, IBM
