On 7/20/06, George Harley wrote:
<SNIP!> Anyway, the point I guess that I am trying to make here is that it is possible in TestNG to select the methods to test dynamically using a little bit of scripting that (a) gives us a lot more power than the include/exclude technique and (b) will work the same across every platform we test on. Because BeanShell allows us to instantiate and use Java objects of any type on the classpath then the possibility of using more than just group membership to decide on tests to run becomes available to us. Please refer to the TestNG documentation for more on the capabilities of BeanShell and the TestNG API. I had never heard of it before never mind used it but still managed to get stuff working in a relatively short space of time. I hope this helps. Maybe I need to write a page on the wiki or something ?
Hi George, It would be great to have your proposal for using TestNG on web-site like we have for "Testing conventions" [1]. Thanks, Stepan. [1] http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html Best regards,
George >> Best regards, >> George >> >> >> >>>> Thanks for reading this far. >>>> >>>> Best regards, >>>> George >>>> >>>> >>>> >>>> George Harley wrote: >>>>> Hi, >>>>> >>>>> Just seen Tim's note on test support classes and it really caught >>>>> my attention as I have been mulling over this issue for a little >>>>> while now. I think that it is a good time for us to return to the >>>>> topic of class library test layouts. >>>>> >>>>> The current proposal [1] sets out to segment our different types >>>>> of test by placing them in different file locations. After looking >>>>> at the recent changes to the LUNI module tests (where the layout >>>>> guidelines were applied) I have a real concern that there are >>>>> serious problems with this approach. We have started down a track >>>>> of just continually growing the number of test source folders as >>>>> new categories of test are identified and IMHO that is going to >>>>> bring complexity and maintenance issues with these tests. >>>>> >>>>> Consider the dimensions of tests that we have ... >>>>> >>>>> API >>>>> Harmony-specific >>>>> Platform-specific >>>>> Run on classpath >>>>> Run on bootclasspath >>>>> Behaves different between Harmony and RI >>>>> Stress >>>>> ...and so on... >>>>> >>>>> >>>>> If you weigh up all of the different possible permutations and >>>>> then consider that the above list is highly likely to be extended >>>>> as things progress it is obvious that we are eventually heading >>>>> for large amounts of related test code scattered or possibly >>>>> duplicated across numerous "hard wired" source directories. How >>>>> maintainable is that going to be ? >>>>> >>>>> If we want to run different tests in different configurations then >>>>> IMHO we need to be thinking a whole lot smarter. We need to be >>>>> thinking about keeping tests for specific areas of functionality >>>>> together (thus easing maintenance); we need something quick and >>>>> simple to re-configure if necessary (pushing whole directories of >>>>> files around the place does not seem a particularly lightweight >>>>> approach); and something that is not going to potentially mess up >>>>> contributed patches when the file they patch is found to have been >>>>> recently pushed from source folder A to B. >>>>> >>>>> To connect into another recent thread, there have been some posts >>>>> lately about handling some test methods that fail on Harmony and >>>>> have meant that entire test case classes have been excluded from >>>>> our test runs. I have also been noticing some API test methods >>>>> that pass fine on Harmony but fail when run against the RI. Are >>>>> the different behaviours down to errors in the Harmony >>>>> implementation ? An error in the RI implementation ? A bug in the >>>>> RI Javadoc ? Only after some investigation has been carried out do >>>>> we know for sure. That takes time. What do we do with the test >>>>> methods in the meantime ? Do we push them round the file system >>>>> into yet another new source folder ? IMHO we need a testing >>>>> strategy that enables such "problem" methods to be tracked easily >>>>> without disruption to the rest of the other tests. >>>>> >>>>> A couple of weeks ago I mentioned that the TestNG framework [2] >>>>> seemed like a reasonably good way of allowing us to both group >>>>> together different kinds of tests and permit the exclusion of >>>>> individual tests/groups of tests [3]. I would like to strongly >>>>> propose that we consider using TestNG as a means of providing the >>>>> different test configurations required by Harmony. Using a >>>>> combination of annotations and XML to capture the kinds of >>>>> sophisticated test configurations that people need, and that >>>>> allows us to specify down to the individual method, has got to be >>>>> more scalable and flexible than where we are headed now. >>>>> >>>>> Thanks for reading this far. >>>>> >>>>> Best regards, >>>>> George >>>>> >>>>> >>>>> [1] >>>>> http://incubator.apache.org/harmony/subcomponents/classlibrary/testing.html >>>>> >>>>> [2] http://testng.org >>>>> [3] >>>>> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200606.mbox/[EMAIL PROTECTED] >>>>>
-- Thanks, Stepan Mishura Intel Middleware Products Division ------------------------------------------------------ Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
